Discussion:
13.0 amd64 packages are out-of-date compared to 13.0 i386
Jan Beich
2021-06-02 13:13:44 UTC
Permalink
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done up to 89.0_2,2.
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2 and 89.0_1,2
are not offered already?

[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search "130amd64")
Ronald Klop
2021-06-02 14:32:58 UTC
Permalink
Van: Jan Beich <***@FreeBSD.org>
Datum: woensdag, 2 juni 2021 15:13
Aan: freebsd-***@freebsd.org
Onderwerp: 13.0 amd64 packages are out-of-date compared to 13.0 i386
Post by Jan Beich
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done up to 89.0_2,2.
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2 and 89.0_1,2
are not offered already?
[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search "130amd64")
It can take a couple of hours before the new packages are copied to the mirror servers.
Your computer does not directly download the pkgs from the build servers.

Regards,
Ronald.
Kevin Oberman
2021-06-03 05:02:18 UTC
Permalink
It can take a lot longer than a couple of hours. For a large update
(>30,000 package builds), it takes about four days to complete the build of
all ports and pushing them out to all of the mirrors can take another 4
days. Some of the links are rather slow and error prone. Until the update
is in sync on all official mirrors, no mirror may distribute the updated
packages. There is a week-old 32,000 port upgrade that started 7 days ago.
I expect it to be available very shortly. and an additional upgrade of
about 5000 packages completed yesterday. It will probably be updated to all
mirrors a few hours after the forest one is available. Another build of
over 31,000 packages started about four hours ago. Look for it to show up
in a bit over a week if all goes well.

It's a big job. I believe chromium takes the most time to build at over 30
hours, though electron is not far behind. The build machine builds 16
packages at a time and it is not unusual to see half of the 16 slots have
been building a single package for over an hour and many take >6 hours.
Things like compilers, especially rust, with all libraries are especially
time consuming. Then the compressed copy of the packages is a LOT of bytes
and, for many locations with slow, error prone connections it simply takes
days to move the data. To quote a fictional character of my youth,
"Patience, Grasshopper."
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: ***@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
Post by Ronald Klop
Datum: woensdag, 2 juni 2021 15:13
Onderwerp: 13.0 amd64 packages are out-of-date compared to 13.0 i386
Post by Jan Beich
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done up
to 89.0_2,2.
Post by Jan Beich
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2
and 89.0_1,2
Post by Jan Beich
are not offered already?
[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search
"130amd64")
It can take a couple of hours before the new packages are copied to the mirror servers.
Your computer does not directly download the pkgs from the build servers.
Regards,
Ronald.
Mark Millard via freebsd-ports
2021-06-02 22:34:30 UTC
Permalink
Ronald Klop <ronald-lists_at_klop.ws> wrote on
Post by Ronald Klop
It can take a couple of hours before the new packages are copied to the mirror servers.
Your computer does not directly download the pkgs from the build servers.
http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/ shows
(in date/time order, not in version number order):

firefox/88.0_2,2/130amd64-default.log 04-May-2021 12:13 14460006

firefox/88.0.1_1,2/130amd64-default.log 22-May-2021 15:05 14659916
firefox/89.0,2/130amd64-default.log 27-May-2021 22:41 14368456
firefox/89.0_1,2/130amd64-default.log 30-May-2021 13:49 14390758

poudriere was still running overall (at the time):
firefox/89.0_2,2/130amd64-default.log 01-Jun-2021 13:41 14371282

Using an example: firefox/89.0,2 was in a build that
went over (start time and elapsed):

Thu, 27 May 2021 01:03:13 GMT 63:08:37

and so finished somewhat before on 2021-May-30 GMT.

The time frame does not fit with "a couple of hours" for it not
having shown up (yet). That is likely an example of why Jan B.
Post by Ronald Klop
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done up to 89.0_2,2.
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2 and 89.0_1,2
are not offered already?
[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search "130amd64")
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Greg Rivers via freebsd-ports
2021-06-02 23:37:45 UTC
Permalink
Post by Mark Millard via freebsd-ports
Ronald Klop <ronald-lists_at_klop.ws> wrote on
Post by Ronald Klop
It can take a couple of hours before the new packages are copied to the mirror servers.
Your computer does not directly download the pkgs from the build servers.
http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/ shows
firefox/88.0_2,2/130amd64-default.log 04-May-2021 12:13 14460006
firefox/88.0.1_1,2/130amd64-default.log 22-May-2021 15:05 14659916
firefox/89.0,2/130amd64-default.log 27-May-2021 22:41 14368456
firefox/89.0_1,2/130amd64-default.log 30-May-2021 13:49 14390758
firefox/89.0_2,2/130amd64-default.log 01-Jun-2021 13:41 14371282
Using an example: firefox/89.0,2 was in a build that
Thu, 27 May 2021 01:03:13 GMT 63:08:37
and so finished somewhat before on 2021-May-30 GMT.
The time frame does not fit with "a couple of hours" for it not
having shown up (yet). That is likely an example of why Jan B.
Post by Ronald Klop
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done up to 89.0_2,2.
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2 and 89.0_1,2
are not offered already?
[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search "130amd64")
Based on pkg metadata, I don't think the default/latest pkg repo has been updated at all for the past 10 days:

$ TZ=UTC ls -lt /var/db/pkg
total 76255
-rw-r--r-- 1 root wheel 90916864 Jun 2 23:25 local.sqlite
-r--r--r-- 1 root wheel 6688991 Jun 1 08:15 vuln.xml
-rw-r--r-- 1 root wheel 158 May 23 14:21 FreeBSD.meta
-rw-r--r-- 1 root wheel 51232768 May 23 14:21 repo-FreeBSD.sqlite

This has happened before[1]. Probably something is preventing the repo from being updated.

[1] <https://lists.freebsd.org/pipermail/freebsd-ports/2020-December/119813.html>
--
Greg
Kevin Oberman
2021-06-03 05:14:35 UTC
Permalink
I have previously confirmed with one of the cluster admins that a build of
that size can take over 4 days to distribute to all mirrors and no mirror
starts distributing updated packages until all mirrors are either in sync
of have been pulled from the list of active mirrors if there is a problem
with one. It's a really huge amount of data and the places that havd
mirrors on slow links are the ones that rally need to load packages form a
fairly local mirror.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: ***@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683


On Wed, Jun 2, 2021 at 4:39 PM Greg Rivers via freebsd-ports <
Post by Ronald Klop
Post by Mark Millard via freebsd-ports
Ronald Klop <ronald-lists_at_klop.ws> wrote on
Post by Ronald Klop
It can take a couple of hours before the new packages are copied to
the mirror servers.
Post by Mark Millard via freebsd-ports
Post by Ronald Klop
Your computer does not directly download the pkgs from the build
servers.
Post by Mark Millard via freebsd-ports
http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/ shows
firefox/88.0_2,2/130amd64-default.log
04-May-2021 12:13 14460006
Post by Mark Millard via freebsd-ports
firefox/88.0.1_1,2/130amd64-default.log
22-May-2021 15:05 14659916
Post by Mark Millard via freebsd-ports
firefox/89.0,2/130amd64-default.log
27-May-2021 22:41 14368456
Post by Mark Millard via freebsd-ports
firefox/89.0_1,2/130amd64-default.log
30-May-2021 13:49 14390758
Post by Mark Millard via freebsd-ports
firefox/89.0_2,2/130amd64-default.log
01-Jun-2021 13:41 14371282
Post by Mark Millard via freebsd-ports
Using an example: firefox/89.0,2 was in a build that
Thu, 27 May 2021 01:03:13 GMT 63:08:37
and so finished somewhat before on 2021-May-30 GMT.
The time frame does not fit with "a couple of hours" for it not
having shown up (yet). That is likely an example of why Jan B.
Post by Ronald Klop
"pkg install firefox" still offers 88.0.1_1,2 but builds[1] were done
up to 89.0_2,2.
Post by Mark Millard via freebsd-ports
Post by Ronald Klop
Ignoring 89.0_2,2 where other packages haven't finished yet why 89.0,2
and 89.0_1,2
Post by Mark Millard via freebsd-ports
Post by Ronald Klop
are not offered already?
[1] http://beefy16.nyi.freebsd.org/data/latest-per-pkg/firefox/
https://pkg-status.freebsd.org/?all=1&type=package (Search
"130amd64")
Based on pkg metadata, I don't think the default/latest pkg repo has been
$ TZ=UTC ls -lt /var/db/pkg
total 76255
-rw-r--r-- 1 root wheel 90916864 Jun 2 23:25 local.sqlite
-r--r--r-- 1 root wheel 6688991 Jun 1 08:15 vuln.xml
-rw-r--r-- 1 root wheel 158 May 23 14:21 FreeBSD.meta
-rw-r--r-- 1 root wheel 51232768 May 23 14:21 repo-FreeBSD.sqlite
This has happened before[1]. Probably something is preventing the repo from being updated.
[1] <
https://lists.freebsd.org/pipermail/freebsd-ports/2020-December/119813.html
--
Greg
Loading...