Repository navigation
exe:cabal doesn't properly ignore .cabal files with unsupported cabal-version: #4624
Description
This is causing problems for e.g. cabal-install-1.24 (and older versions as well), which fails to parse bleeding edge .cabal files.
So e.g.
cabal-1.24 info base-noprelude-4.10.0.0
succeeds, even though base-noprelude-4.10.0.0 would require cabal-2.0 to be properly decodable.
But there's more harmful cases, like e.g. for
cabal-1.24 info haddock-library
cabal: internal error when reading package index: failed to parse .cabal file
The package index or index cache is probably corrupt. Running cabal update might fix it.
or even cabal-1.24 list which fails as soon as such a .cabal file is encountered.
Or even worse, cause breakages such as in jgm/pandoc#3814 which render cabal-1.24 unusable.
In the past this would have been mitigated by having cabal update remind users to update, e.g.
$ cabal-1.20 update
Downloading the latest package list from hackage.haskell.org
Skipping download: Local and remote files match.
Note: there is a new version of cabal-install available.
To upgrade, run: cabal install cabal-install
How to fix (imho)
Quickfix
- ignore
.cabalfiles in the index we can't parse (i.e. which the parser currently fails with an "internal error") - possibly, for those
.cabalfiles we can parse but have a too-newcabal-version, ignore them as well
(see also #4624 (comment))
Proper fix
For packages that are newer than cabal-install supports, we can only know the package name & version, and possibly the specified cabal-version. Everything beyond that would require the parser to know how to parse and interpret the .cabal contents (note that semantics may change depending on the cabal-version, so we really cannot know what a .cabal files means if we don't actively support the declared cabal-version).
I've looked at the code in D.C.IndexUtils, and one way to go about this is to do something along the lines of
--- a/cabal-install/Distribution/Client/IndexUtils.hs
+++ b/cabal-install/Distribution/Client/IndexUtils.hs
@@ -56,7 +56,7 @@ import Distribution.Simple.Program
import qualified Distribution.Simple.Configure as Configure
( getInstalledPackages, getInstalledPackagesMonitorFiles )
import Distribution.ParseUtils
- ( ParseResult(..) )
+ ( ParseResult(..), PError(FromString) )
import Distribution.Version
( Version(Version), intersectVersionRanges )
import Distribution.Text
@@ -254,7 +254,7 @@ whenCacheOutOfDate index action = do
-- | An index entry is either a normal package, or a local build tree reference.
data PackageEntry =
- NormalPackage PackageId GenericPackageDescription ByteString BlockNo
+ NormalPackage PackageId (Maybe GenericPackageDescription) ByteString BlockNo
| BuildTreeRef BuildTreeRefType
PackageId GenericPackageDescription FilePath BlockNo
@@ -541,13 +542,16 @@ packageListFromCache mkPkg hnd Cache{..} mode = accum mempty [] cacheEntries
-> return content
_ -> interror "unexpected tar entry type"
- readPackageDescription :: ByteString -> IO GenericPackageDescription
+ readPackageDescription :: ByteString -> IO (Maybe GenericPackageDescription)
readPackageDescription content =
case parsePackageDescription . ignoreBOM . fromUTF8 . BS.Char8.unpack $ content of
- ParseOk _ d -> return d
- _ -> interror "failed to parse .cabal file"
+ ParseOk _ d -> return (Just $! d)
+ ParseFailed (FromString e _)
+ | "This package requires at least Cabal version" `isPrefixOf` e
+ -> return Nothing
+ _ -> interror "failed to parse .cabal file"I.e. when a GenericPackageDescription cannot be parsed, turn it into a Nothing (or something with more information, providing more details about the minimum required version).
Then we could have the solver handle "future" packages by placing them into a blacklist
(as if they had an unconditional fail: this package requires a newer cabal version, see #393) - or we could just fake a minimal GenericPackageDescription which consists merely of an empty lib and exe with such a fail directive. Actually, would it work already now if we just faked a dummy GenericPackageDescription with a too-new spec-cabal-version?
Anyway, then the solver would avoid selecting such packages, and if forced to (freeze file, --constraints, or merely cabal install foo-1.2.3), would present the user with an informative error message which points to a possible resolution.
Commands like cabal info or cabal list would have to gracefully skip, ignore, or if forced to (cabal info foo-1.2.3), emit useful messages.
/cc @GrayJay
Activity
jgm commentedon Jul 24, 2017
Note that this also seems to affect cabal-1.22 and cabal-1.18, as the pandoc travis builds show.
hvr commentedon Jul 24, 2017
Indeed. We may require to release hotfixes to a couple of older major versions of
cabal-install, so Linux distributions can pick it up./cc @23Skidoo
23Skidoo commentedon Jul 24, 2017
:-(
I'll try to fix this (and the remaining exe:cabal-install-2.0 blockers) during this week.
hvr commentedon Jul 25, 2017
@23Skidoo here's an alternative & complementary idea: older cabals (i.e. cabal-install-1.22 and older) only know about the 00-index.tar.gz; so we could remove cabal-version:>=2.0 files from the 00-index.tar.gz server-side; so only 01-index.tar.gz would contain .cabal files requiring cabal>=2. I've looked at hackage-server's code, and this requires a bit of work to get implemented...
Then we'd only need a quickfix for cabal-install-1.24, and a proper fix for cabal-install-2.0+ (so we won't run into this when we extend the .cabal format again in 2.2)
Work around haskell/cabal#4624
sol commentedon Jul 26, 2017
@RyanGlScott I'm not sure if ekmett/semigroupoids@ef085d1 is a viable solution. As I understand it masks the problem on Travis, but users that use a released version of
cabal-installare still affected.@hvr for now I'm avoiding to run
cabal update. Which packages on Hackage cause this issue? Wouldn't we want to blacklist (or revise) those packages for now?gallais commentedon Jul 26, 2017
Yesterday I ran a small script untarring and grepping for these
cabal-version: >=2.0constraints and the packages having them were: haddock-api, haddock-library, haddock, and base-nopreludeezyang commentedon Jul 27, 2017
I don't think this is a blocker for 2.0. We can always get the fix which solves
cabal-version: >= 2.2into cabal-install-2.0 into a bugfix release before Cabal-2.2 comes out. In the meantime, not having a cabal-install-2.0 means that it's not possible to work around this bug by installing the latest cabal-install.RyanGlScott commentedon Jul 27, 2017
@sol, to be clear, I don't intend for ekmett/semigroupoids@ef085d1 to be a permanent fix. I only committed that to repair Travis breakage after a previous commit I made, and I needed to bandage the wound to save face.
That being said, I've pretty much stopped making commits to most of my Haskell projects until this issue is fixed so as not to break things further.
hvr commentedon Jul 28, 2017
@RyanGlScott et al., I've deployed a server-side workaround for legacy cabal-install clients (i.e. before 2.0). Can you give it a try and see if this fixes more than it breaks?
Revert "Work around haskell/cabal#4624"
Revert "[ #2655 ] Temporally disabled `make TAGS` with GHC 7.8.4. (#2…
15 remaining items
23Skidoo commentedon Sep 7, 2017
I'm also not too hot on the grace period idea, but that's more of a
hackage-serverissue, so let's keep the discussion on-topic.bgamari commentedon Oct 6, 2017
What is the status of this? The next GHC release candidate will likely be in a few weeks and I would prefer to avoid submodule bumps after it goes out.
hvr commentedon Oct 6, 2017
@bgamari I've got a working proof of concept, and it doesn't require changes to lib:Cabal; you should however keep an eye on the #4808 regression which can be considered a release-blocker for Cabal-2.0.1
23Skidoo commentedon Dec 8, 2017
@hvr I guess this can be closed in favour of #4899?
hvr commentedon Dec 8, 2017
@23Skidoo Indeed; everything's been put in place for the officially released cabal versions; it's just cabal-2.2 that needs a little bit of work.
hackage: download legacy index instead of symlink