Enhancement: kpm requires offline mode to solve the situation without network

[zong-zhe]

Enhancement

Currently, kpm needs to connect to the network to get the metadata of the kcl dependencies, but in some cases, the kpm can not access to the network, so it is necessary to add offline mode to support the operation of kpm in a no-network environment.

[vinayakjaas]

I have some question regarding to this issue
1.Which specific functionalities of kpm need to be available in offline mode (e.g., dependency installation, updates, package initialization)?
2.What specific metadata needs to be cached?
I go through this type of issue in different project as well where they applied Caching mechanism to store metadata so can we applied this issue ? .Just a suggestion I not no weather it`s work or not because I not able to understand this issue in detail.

[zong-zhe]

Hi @vinayakjaas 😃

Good question! Let me provide a few more details on this issue.

1. A common working pattern of package management tools is to store the metadata of all packages in a specific repository. Before downloading a package, the package management tool needs to request package information from the metadata repository and then download the corresponding content according to the specific information of the package.

2. At present, the main official package store of kpm is the OCI registry ghcr.io and docker.io, which stores the metadata of each package through an OCI manifest. Each time a package is requested from the OCI registry, the OCI manifest needs to be requested first, followed by the package's content. The problem here is that the OCI registry needs to be requested twice, which can result in poor performance due to network issues.

3. The common solution for 2 is 1 above, just storing all metadata through a unified repository and cache it locally for users. Each time the package management tool is started, it needs to update the metadata cached locally by users, and then pull the three-party library based on the metadata.

4. offine mode will close the auto update of the metadata cached locally for users.

The current lead work for this job has not been completed, and the related lead work is tracked by:

• [LFX Term 3]: Full storage path for KCL third-party dependencies #384
• [LFT Term 3]: The checksum check of the three-party dependencies #394

[ravjot07]

Hey @zong-zhe ,

I've been diving into the details of the offline mode for KPM and I'm really interested in how this setup will handle checksum verification, which is crucial for maintaining package integrity and security without network access.

Here are a couple of thoughts and questions I have:

Checksum Verification Process: How will the offline mode affect the checksum verification for downloaded packages? I'm thinking we could implement a mechanism that checks checksums against a pre-cached list to ensure integrity even when offline. It could be a good enhancement for the LFX checksum project.

Fallback Mechanisms: If checksum verification fails while offline, what fallback processes do you think we should propose to keep the system reliable and secure? I want to make sure we're heading in the right direction with this part of the project.