hal_adi: Synchronization problem

[hfakkiz]

Describe the issue
Currently, we create "zephyrproject-rtos/hal_adi" commits manually regarding to features dependencies which we want to add Zephyr. This causes a synchronization problem between inner "hal_adi" and "zephyrproject-rtos/hal_adi" repositories.

Expected behavior
Having linear history, same as on inner "hal_adi" repository.

Impact
It is hard to synchronize.

Steps to resolve

1. Add zephyrproject-rtos/hal_adi/main commit with information that "hal_adi" history will be rewritten and the main branch will be pushed with new history; merge it and tag.
2. Tag and branch to use: zephyr-v4.0.0-snapshot
3. Push new-history branch with fixed history;
4. Post Zephyr PR referencing the "hal_adi" from the new-history branch by the sha.
5. Merge the Zephyr PR fixing the "hal_adi"
6. Overwrite the main branch with new-history branch.

[github-actions]

Hi @hfakkiz! We appreciate you submitting your first issue for our open-source project. 🌟

Even though I'm a bot, I can assure you that the whole community is genuinely grateful for your time and effort. 🤖💙

[nashif]

This is not a Zephyr bug, is it?

[hfakkiz]

This is not a Zephyr bug, is it?

No, it is not a Zephyr bug. It's about "zephyrproject-rtos/hal_adi" repository.

[MaureenHelm]

This is not a Zephyr bug, is it?

The hal_adi module is having the same problem we previously had with mcuboot, where the Zephyr fork of the module repo got out of sync with the upstream module repo. I asked @hfakkiz to create an issue following #55303 as an example to document the problem and how he wants to fix it.

[fabiobaltieri]

May attach it to #85322 if we end up having a process to rework repos

[petejohanson-adi]

Ok, posting this slightly lengthy proposal to get feedback, regarding how we'd like to fix hal_adi specifically:

hal_adi Refresh Overview

The zephyrproject-rtos/hal_adi has several key problems as it exists today:

• It is "disconnected" from the upstream real source of truth, analogdevicesinc/hal_adi
• The branch commit sequence has diverged from upstream

As a result, fixes and enhancements to the Zephyr version of the ADI HAL has required manual cherry-picking of commits and PRs, resulting in additional maintenance burden and confusion.

MSDK Export Process Improvements

The analogdevicesinc/hal_adi repository is a selective export of the MSDK (analogdevicesinc/msdk). The exported version has had occasional issues with the generated export, and as part of the preparation for the Zephyr maintenance work, ADI has created an improved export process that better preserves the upstream commit history. That new branch in analogdevicesinc/hal_adi named msdk-export, is intended as the new, canonical MSDK export for use in Zephyr. Comparison of the content of that branch compared to the existing branch, along with internal testing, has been used to verify the new export's quality and readiness for use.

Proposal

ADI would like to perform a one-time maintenance operation to replace the current zephyrproject-rtos/hal_adi with a connected fork of analogdevicesinc/hal_adi, using the new msdk-export branch therein. This will allow easy updates to the Zephyr version of the ADI HAL by using the "Sync Fork" action within the GitHub UI, or via future CI automation.

The process will leverage GitHub's behavior of automatic redirects after a rename in order to swap in the new fork without interruption to consumers running west update. All commits in the existing zephyrproject-rtos/hal_adi will be pushed into analogdevicesinc/hal_adi before the move, ensuring that after the change over, previous versions of Zephyr that reference those commits will continue to work as expected.

Risks

The following risks exist with the sequence proposed here:

• Subtle differences between Zephyr's GitHub organization and the one used for ADI's internal testing may cause problems.
• GitHub's automatic redirect mechanism when a repository is renamed to "take over" from an existing redirect is not 100% atomic, and any in-flight git operations acessing the "old" repo via the redirect are interrupted or have undefined behavior.
• Some other unforeseen issue

Exact Sequence

Roles (may be the same person):

• ADI Admin - an Analog Devices, Inc employee with admin/maintainer access
• Zephyr Admin - a Zephyr contributor with admin/maintainer access

1. ADI Admin pushes all branches/tags from zephyrproject-rtos/hal_adi to analogdevicesinc/hal_adi, with naming convention to clearly separate out their origin/purpose. As a safety check, they locally change the remote for the HAL ADI in the Zephyr west.yml file and verify that a west update works as expected. This chech to be peformed against several Zephyr release tags and main.
2. Zephyr Admin uses the GitHub UI to rename zephyrproject-rtos/hal_adi to zephyrproject-rtos/hal_adi-archive
3. Zephyr/ADI Admin verifies west update functionality is unaffected, as a safety checkpoint. If any issue are encountered, the zephyrproject-rtos/hal_adi-archive will be renamed back to zephyrproject-rtos/hal_adi and the procedure will be halted.
4. Zephyr Admin uses the GitHub UI to fork the analogdevicesinc/hal_adi repo into the Zephyr organization at zephyrproject-rtos/hal_adi.
5. Zephyr/ADI Admin verifies west update functionality is unaffected, as a safety checkpoint. If any issue are encountered, the zephyrproject-rtos/hal_adi will be renamed to zephyrproject-rtos/hal_adi-new-fork, the zephyrproject-rtos/hal_adi-archive will be renamed back to zephyrproject-rtos/hal_adi and the procedure will be halted.
6. Zephyr Admin archives the old zephyrproject-rtos/hal_adi-archive repository with the GitHub UI.
7. Zephyr/ADI Admin verifies west update functionality is unaffected, as a final safety checkpoint.

Alternatives

The following alternatives were considered:

Manual Updates

The new MSDK export branch could be manually pushed periodically into a new branch of the existing zephyrproject-rtos/hal_adi repo. This would require sufficient access to that repo, management of said access, and a manual process that may introduce errors.

Automation of Manual Sync

Some PoC work exists that manually syncs repo contents, which was developed to address a similar issue with the Zephyr LVGL repo: #85322 (comment)

We could forgo the rename/fork operation in favor of the less risky, but more setup/maintenance effort of adding that automation.