Last modified: 2014-09-19 20:03:16 UTC
on instance wikistats-petcow, major puppet fail related to package installs, without having changed it and with it working just fine before: Sep 18 01:04:32 wikistats-petcow puppet-agent[347]: E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:32 wikistats-petcow puppet-agent[347]: E: The package lists or status file could not be parsed or opened. Sep 18 01:04:32 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Standard-packages/Package[xfsprogs]/ensure) change from 3.1.9ubuntu2 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy xfsprogs' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:32 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Standard-packages/Package[xfsprogs]/ensure) E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:32 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Standard-packages/Package[xfsprogs]/ensure) E: The package lists or status file could not be parsed or opened. Sep 16 21:31:54 wikistats-petcow crontab[734]: (root) LIST (root) Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: Could not get latest version: Execution of '/usr/bin/apt-cache policy ntp' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: E: The package lists or status file could not be parsed or opened. Sep 16 21:31:54 wikistats-petcow crontab[737]: (root) LIST (wikistatsuser) Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: (/Stage[main]/Ntp/Package[ntp]/ensure) change from 1:4.2.6.p5+dfsg-3ubuntu2 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy ntp' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: (/Stage[main]/Ntp/Package[ntp]/ensure) E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:34 wikistats-petcow puppet-agent[347]: (/Stage[main]/Ntp/Package[ntp]/ensure) E: The package lists or status file could not be parsed or opened. Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: Could not get latest version: Execution of '/usr/bin/apt-cache policy arcconf' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: E: The package lists or status file could not be parsed or opened. Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Monitoring::Host/Package[arcconf]/ensure) change from 7.31.18856-1 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy arcconf' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Monitoring::Host/Package[arcconf]/ensure) E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:35 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Monitoring::Host/Package[arcconf]/ensure) E: The package lists or status file could not be parsed or opened. Sep 18 01:04:37 wikistats-petcow puppet-agent[347]: Could not get latest version: Execution of '/usr/bin/apt-cache policy megacli' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:37 wikistats-petcow puppet-agent[347]: E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages Sep 18 01:04:37 wikistats-petcow puppet-agent[347]: E: The package lists or status file could not be parsed or opened. Sep 18 01:04:37 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Monitoring::Host/Package[megacli]/ensure) change from 8.07.14-1 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy megacli' returned 100: E: Encountered a section with no Package: header Sep 18 01:04:37 wikistats-petcow puppet-agent[347]: (/Stage[main]/Base::Monitoring::Host/Package[megacli]/ensure) E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages and a LOT more of those, basically one for each and every package it's trying to install i'm using labsdebrepo here, but it did not cause this problem before at all
doesn't go away after de-selecting "labsdebrepo" role either: root@wikistats-petcow:/root# file _data_project_repo_Packages _data_project_repo_Packages: gzip compressed data, gunzip data.gzip gzip: data.gzip: unknown suffix -- ignored --- Error: Could not get latest version: Execution of '/usr/bin/apt-cache policy acct' returned 100: E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages E: The package lists or status file could not be parsed or opened. Error: /Stage[main]/Base::Standard-packages/Package[acct]/ensure: change from 6.5.5-1ubuntu5 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy acct' returned 100: E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages E: The package lists or status file could not be parsed or opened. Error: Could not get latest version: Execution of '/usr/bin/apt-cache policy htop' returned 100: E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages E: The package lists or status file could not be parsed or opened. Error: /Stage[main]/Base::Standard-packages/Package[htop]/ensure: change from 1.0.2-3 to latest failed: Could not get latest version: Execution of '/usr/bin/apt-cache policy htop' returned 100: E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/_data_project_repo_Packages E: The package lists or status file could not be parsed or opened.
(In reply to Daniel Zahn from comment #1) > doesn't go away after de-selecting "labsdebrepo" role either: > > root@wikistats-petcow:/root# file _data_project_repo_Packages > _data_project_repo_Packages: gzip compressed data, > renamed to .gz, unpacked it. the content is my package i had formerly added to labsdebrepo. somehow this is a change that conflicts with the former labsdebrepo setup
deleting /var/lib/apt/lists/_data_project_repo_Packages doesn't fix this either and i don't see any change that uses "labsdeb" in the commit message and looks new
I had a similar issue on tools yesterday; something seems to have changed in or around apt-get such that if the repo's list is compressed (Packages.gz) then apt-get update gets confused and caches it compressed - but it expects the local caches to be /un/compressed. It was fixed on tools by /not/ compressing the package list (that is, the output of dpkg-scanpackages is saved, uncompressed, to "Packages"). I'm not entirely clear on what caused this change; it doesn't seem to come from puppet at all.
(In reply to Marc A. Pelletier from comment #4) > It was fixed on tools by /not/ compressing the package list (that is, the > output of dpkg-scanpackages is saved, uncompressed, to "Packages"). it's gzipped, so i moved it to *.gz , gunzipped it.. but on next run of apt-get update or puppet agent -tv i always get a new compressed version of the file and the error stays the same. is there a patch you made to fix it on toollabs you could link to?
(In reply to Marc A. Pelletier from comment #4) > It was fixed on tools by /not/ compressing the package list (that is, the > output of dpkg-scanpackages is saved, uncompressed, to "Packages"). oh yea, thanks. confirmed. unzipping /data/project/repo/Packages.gz did fix it indeed. just a bit confusing that the actual error is about /var/lib/apt/lists/_data_project_repo_Packages and where this change came from
Yeah, the error message unhelpfully points to the locally cached version and not the source. I'm still unsure what /caused/ this change of behaviour; though. It seems odd that it would still /accept/ Packages.gz if it does not actually process them anymore. It may well be an upstream bug, but I'd have thought something this highly visible would have been noticed.
btw, i wonder if we should merge https://gerrit.wikimedia.org/r/#/c/145573/ anyways. that change looks good to me, but this very bug is kind of related because it checks for Packages.gz
