mirror download.opensuse.com not working

[kisahm]

mirroring a debian repository from download.opensuse.com is not working.

Detailed Description

aptly doesn't find items in remote packages.gz

# aptly mirror create crio-1.19 https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/ ""
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/InRelease...
gpgv: Signature made Mon 21 Dec 2020 04:53:39 PM CET using RSA key ID 75060AA4
gpgv: Good signature from "devel:kubic OBS Project <devel:[email protected]>"

Mirror [crio-1.19]: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/ ./ successfully added.
You can run 'aptly mirror update crio-1.19' to download repository contents.

# aptly mirror update crio-1.19
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/InRelease...
gpgv: Signature made Mon 21 Dec 2020 04:53:39 PM CET using RSA key ID 75060AA4
gpgv: Good signature from "devel:kubic OBS Project <devel:[email protected]>"
Downloading & parsing package files...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Packages.gz...
Building download queue...
Download queue: 0 items (0 B)

Mirror `crio-1.19` has been successfully updated.

Packages.gz contains the items:

# wget -O - https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Packages.gz|gunzip
--2021-01-04 12:06:00--  https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Packages.gz
Resolving download.opensuse.org (download.opensuse.org)... 195.135.221.134, 2001:67c:2178:8::13
Connecting to download.opensuse.org (download.opensuse.org)|195.135.221.134|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3448 (3.4K) [application/x-gzip]
Saving to: ‘STDOUT’

-                                                  100%[================================================================================================================>]   3.37K  --.-KB/s    in 0s      

2021-01-04 12:06:00 (133 MB/s) - written to stdout [3448/3448]

Package: cri-o
Version: 1.19.0~0
Architecture: amd64
Maintainer: Douglas Schilling Landgraf <[email protected]>
Installed-Size: 87104
Depends: libgpgme11, libseccomp2, conmon, containers-common (>= 0.1.27), runc | cri-o-runc (>= 1.0.0)
Suggests: containernetworking-plugins
Replaces: cri-o-1.10, cri-o-1.11, cri-o-1.11-stable, cri-o-1.12, cri-o-1.13, cri-o-1.14, cri-o-1.15, cri-o-1.16, cri-o-1.17
Filename: ./amd64/cri-o_1.19.0~0_amd64.deb
Size: 18674480
MD5sum: 49d18fe4fa82d2f4777099552aa806e8
SHA1: eb63afb721dfa4b81d2b93fdbb05556f3c81588d
SHA256: 86498651386c5bfcd4a28b1fb8bd281978b7e1ab3f0810771259a00fcb9774ca
Section: devel
Priority: optional
Homepage: https://github.com/cri-o/cri-o
Description: OCI-based implementation of Kubernetes Container Runtime Interface.
....

Context

I'm using aptly to manage the repositories for Debian and Ubuntu to create "frozen" releases.

Possible Implementation

problem could be the non-standard structure of the remote repository.
an aptly debug mode would be helpful to see whats do going on.

Your Environment

Debian 10

[RustyDust]

Since I just had the exact same problem with another repo, you might want to try

# aptly mirror create crio-1.19 https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19 ./Debian_10/

to define your remote repository. Note the change in the url and the distribution. As a simple rule of thumb: use the last part of the URL path as your distribution and enclose the name with ./ and /
Works for me - your mileage may vary.

[kisahm]

looks like "Debian_10" as distribution breaks the package link in Packages file.

Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/InRelease...
gpgv: Signature made Mon 04 Jan 2021 06:15:47 PM CET using RSA key ID 75060AA4
gpgv: Good signature from "devel:kubic OBS Project <devel:[email protected]>"

Mirror [test-crio-1.19]: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/ ./Debian_10/ successfully added.
You can run 'aptly mirror update test-crio-1.19' to download repository contents.

# aptly mirror update test-crio-1.19
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/InRelease...
gpgv: Signature made Mon 04 Jan 2021 06:15:47 PM CET using RSA key ID 75060AA4
gpgv: Good signature from "devel:kubic OBS Project <devel:[email protected]>"
Downloading & parsing package files...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Packages.gz...
Building download queue...
Download queue: 1 items (16.62 MiB)
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/i386/cri-o_1.19.0~0_i386.deb...
ERROR: unable to update: download errors:
  HTTP code 404 while fetching https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/i386/cri-o_1.19.0~0_i386.deb

[RustyDust]

I just tried with yet another aptly mirror create syntax which seems to work for the repo you're trying to mirror:

# aptly mirror create test https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10 ./
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Release...

Mirror [test]: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/ ./ [src] successfully added.
You can run 'aptly mirror update test' to download repository contents.
root@rnultra6:/data/mirror# aptly mirror update test
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Release...
Downloading & parsing package files...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Packages.gz...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/Sources.gz...
Building download queue...
Download queue: 23 items (2.92 GiB)
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/i386/golang-1.14-src_1.14.4-3_i386.deb...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/i386/golang-1.15-go_1.15.2-1_i386.deb...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/golang-1.15_1.15.2-1.dsc...
Downloading https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/golang-1.15_1.15.2-1.tar.gz...
 1.03 GiB / 2.92 GiB [=======================================================>------------------------------------------------------------------------------------------------------]  35.33% 12.05 MiB/s 2m39s
 ...

[kisahm]

your're right. now its working....
strange

i need to check my history to find the difference.

thank you.