Generated PURL is not providing enough information to derive a valid download URL

[DennisClark]

Here is a valid download URL: https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz

I successfully created a Package in DejaCode (see attached screenshot) using that download URL and it assigned this PURL to that package: pkg:github/apache/nifi@2.0.0-M3

In a different dataspace, I attempted to create a Package using that PURL, and got this error message:
Error: Could not download content: https://github.com/apache/nifi/archive/2.0.0-M3.tar.gz

so it seems that the PURL does not have the information required to derive the complete, valid download URL.

[tdruez]

>>> from packageurl.contrib import purl2url, url2purl

>>> download_url = "https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz"
>>> print(url2purl.get_purl(download_url))
pkg:github/apache/nifi@2.0.0-M3

>>> purl = "pkg:github/apache/nifi@2.0.0-M3"
>>> purl2url.get_download_url(purl)
'https://github.com/apache/nifi/archive/2.0.0-M3.tar.gz'

The generated download URL is invalid as the tag value is rel/nifi-2.0.0-M3 but it is lost when generating the purl.
We cannot construct https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.zip from just pkg:github/apache/nifi@2.0.0-M3.

In a way, the generated purl is incorrect as the version 2.0.0-M3 do not exist in GitHub, see https://github.com/apache/nifi/releases
It only exists there as the rel/nifi-2.0.0-M3 tag https://github.com/apache/nifi/tags

Maybe the correct purl here should be pkg:github/apache/nifi@rel/nifi-2.0.0-M3

>>> purl2url.get_download_url("pkg:github/apache/nifi@rel/nifi-2.0.0-M3")
'https://github.com/apache/nifi/archive/rel/nifi-2.0.0-M3.tar.gz'   # This URL works!

Alternatively, we may want to store the tag as a qualifier, such as:
pkg:github/apache/nifi@2.0.0-M3?tag=rel/nifi-2.0.0-M3
But this would require additional support in the packageurl library.

[pombredanne]

for https://github.com/apache/nifi/releases/tag/rel%2Fnifi-1.27.0 where the tag is effectively a weird rel/nifi-1.27.0 and same for rel/nifi-2.0.0-M3. We should IMHO use this as a version, e.g., use the tag exactly as it is and not trying to change it in anyway.

>>> p=PackageURL.from_string("pkg:github/apache/nifi@rel/nifi-1.27.0")
>>> p
PackageURL(type='github', namespace='apache', name='nifi', version='rel/nifi-1.27.0', qualifiers={}, subpath=None)
>>> print(p)
pkg:github/apache/nifi@rel/nifi-1.27.0

For the m3 tag, we should have this (and this is NOT what works today):

>>> from packageurl.contrib import purl2url, url2purl

>>> download_url = "https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz"
>>> print(url2purl.get_purl(download_url))
pkg:github/apache/nifi@rel/nifi-2.0.0-M3

>>> purl = "pkg:github/apache/nifi@2.0.0-M3"
>>> purl2url.get_download_url(purl)
'https://github.com/apache/nifi/archive/2.0.0-M3.tar.gz' # <-- this does not exist alright
                                                         # this tag does NOT exist

# this would work with  proper URL, this does exist alright
>>> purl = "pkg:github/apache/nifi@rel/nifi-2.0.0-M3"
>>> purl2url.get_download_url(purl)
`https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz`

So to recap:

1. the url2purl is faulty
2. we should always take the tags as-is, even if rel/ or v prefixed
3. to allow relating a package repo tag to a package version is something entirely different like in purl2vcs in purldb

[tdruez]

PR @ package-url/packageurl-python#161

[tdruez]

Fix merged and deployed.

[DennisClark]

Fix confirmed. Thanks @tdruez !