Skip to content

fix(nix): fix flake lock refresh #33991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

fix(nix): fix flake lock refresh #33991

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0ba64df
fix(nix): flake lock refresh
SuperSandro2000 Feb 1, 2025
1270283
fix(nix): handle nixpkgs upgrades with not explicit ref correct
SuperSandro2000 Feb 5, 2025
f7f2d80
Merge remote-tracking branch 'upstream/main' into fix-nix
SuperSandro2000 Feb 5, 2025
87ed572
Merge remote-tracking branch 'upstream/main' into fix-nix
SuperSandro2000 Mar 11, 2025
27ac0bb
Remove fileMatch hack
SuperSandro2000 Mar 11, 2025
f392715
Force lock file updates for nix
SuperSandro2000 Mar 12, 2025
49525d6
Support new lockable channel
SuperSandro2000 Mar 12, 2025
06a28b4
Fix lints
SuperSandro2000 Mar 12, 2025
9243254
Merge remote-tracking branch 'upstream/main' into fix-nix
SuperSandro2000 Apr 11, 2025
f59711a
Remove hack and set rangeStrategy instead
SuperSandro2000 Apr 11, 2025
c98d086
Fix tests
SuperSandro2000 Apr 11, 2025
499c353
Merge remote-tracking branch 'upstream/main' into fix-nix
SuperSandro2000 Apr 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 174 additions & 12 deletions lib/modules/manager/nix/extract.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,17 +72,18 @@ describe('modules/manager/nix/extract', () => {

it('includes nixpkgs input with no explicit ref', async () => {
fs.readLocalFile.mockResolvedValueOnce(flake1Lock);
expect(await extractPackageFile(flake4Nix, 'flake.nix')).toEqual({
deps: [
{
currentValue: undefined,
datasource: 'git-refs',
depName: 'nixpkgs',
packageName: 'https://github.com/NixOS/nixpkgs',
versioning: 'nixpkgs',
},
],
});
expect(await extractPackageFile(flake4Nix, 'flake.nix')).toBeNull();
});

const flake5Nix = `{
inputs = {
nixpkgs-lib.url = "https://github.com/NixOS/nixpkgs/archive/072a6db25e947df2f31aab9eccd0ab75d5b2da11.tar.gz";
};
}`;

it('includes nixpkgs input with only ref', async () => {
fs.readLocalFile.mockResolvedValueOnce(flake1Lock);
expect(await extractPackageFile(flake5Nix, 'flake.nix')).toBeNull();
});

it('returns null when no inputs', async () => {
Expand Down Expand Up @@ -124,10 +125,11 @@ describe('modules/manager/nix/extract', () => {
deps: [
{
depName: 'nixpkgs',
currentDigest: '9f4128e00b0ae8ec65918efeba59db998750ead6',
currentValue: 'nixos-unstable',
currentDigest: '9f4128e00b0ae8ec65918efeba59db998750ead6',
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -169,6 +171,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'nixpkgs',
packageName: 'https://github.com/NixOS/nixpkgs',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -228,6 +231,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'patchelf',
packageName: 'https://github.com/NixOS/patchelf.git',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -270,6 +274,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'ijq',
packageName: 'https://git.sr.ht/~gpanders/ijq',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -312,6 +317,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'home-manager',
packageName: 'https://gitlab.com/rycee/home-manager',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -368,6 +374,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'nixpkgs',
packageName: 'https://github.com/NixOS/nixpkgs',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -496,6 +503,7 @@ describe('modules/manager/nix/extract', () => {
depName: 'nixpkgs-extra-pkgs',
packageName:
'https://github.corp.example.com/my-org/nixpkgs-extra-pkgs',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -598,6 +606,7 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'data-mesher',
packageName: 'https://git.clan.lol/clan/data-mesher',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down Expand Up @@ -639,6 +648,159 @@ describe('modules/manager/nix/extract', () => {
datasource: 'git-refs',
depName: 'subgroup-project',
packageName: 'https://gitlab.com/group/sub-group/subgroup-project',
rangeStrategy: 'update-lockfile',
},
],
});
});

const flake13Lock = `{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move inside the test and use codeBlock string template helper

"nodes": {
"nixpkgs-lib": {
"locked": {
"lastModified": 1738452942,
"narHash": "sha256-vJzFZGaCpnmo7I6i416HaBLpC+hvcURh/BQwROcGIp8=",
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/072a6db25e947df2f31aab9eccd0ab75d5b2da11.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/072a6db25e947df2f31aab9eccd0ab75d5b2da11.tar.gz"
}
},
"root": {
"inputs": {
"nixpkgs-lib": "nixpkgs-lib"
}
}
},
"root": "root",
"version": 7
}`;

it('includes flake with only tarball type', async () => {
fs.readLocalFile.mockResolvedValueOnce(flake13Lock);
expect(await extractPackageFile('', 'flake.nix')).toBeNull();
});

const flake14Lock = `{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

"nodes": {
"flake-parts": {
"inputs": {
"nixpkgs-lib": "nixpkgs-lib"
},
"locked": {
"lastModified": 1733312601,
"narHash": "sha256-4pDvzqnegAfRkPwO3wmwBhVi/Sye1mzps0zHWYnP88c=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "205b12d8b7cd4802fbcb8e8ef6a0f1408781a4f9",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "flake-parts",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1734649271,
"narHash": "sha256-4EVBRhOjMDuGtMaofAIqzJbg4Ql7Ai0PSeuVZTHjyKQ=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "d70bd19e0a38ad4790d3913bf08fcbfc9eeca507",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs-lib": {
"locked": {
"lastModified": 1733096140,
"narHash": "sha256-1qRH7uAUsyQI7R1Uwl4T+XvdNv778H0Nb5njNrqvylY=",
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/5487e69da40cbd611ab2cadee0b4637225f7cfae.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/5487e69da40cbd611ab2cadee0b4637225f7cfae.tar.gz"
}
},
"root": {
"inputs": {
"flake-parts": "flake-parts",
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}`;

it('includes flake with nixpkgs-lib as tarball type', async () => {
fs.readLocalFile.mockResolvedValueOnce(flake14Lock);
expect(await extractPackageFile('', 'flake.nix')).toMatchObject({
deps: [
{
currentDigest: '205b12d8b7cd4802fbcb8e8ef6a0f1408781a4f9',
currentValue: undefined,
datasource: 'git-refs',
depName: 'flake-parts',
packageName: 'https://github.com/hercules-ci/flake-parts',
rangeStrategy: 'update-lockfile',
},
{
currentDigest: 'd70bd19e0a38ad4790d3913bf08fcbfc9eeca507',
currentValue: 'nixos-unstable',
datasource: 'git-refs',
depName: 'nixpkgs',
packageName: 'https://github.com/nixos/nixpkgs',
rangeStrategy: 'update-lockfile',
},
],
});
});

const flake15Lock = `{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 315532800,
"narHash": "sha256-OBkwS4XoKsUwM8ykjEN1JLg/SI/SHOVJbrwMfK64BJo=",
"type": "tarball",
"url": "https://releases.nixos.org/nixpkgs/nixpkgs-25.05pre766138.b62d2a95c72f/nixexprs.tar.xz"
},
"original": {
"type": "tarball",
"url": "https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}`;

it('includes flake with nixpkgs channel as tarball type', async () => {
fs.readLocalFile.mockResolvedValueOnce(flake15Lock);
expect(await extractPackageFile('', 'flake.nix')).toMatchObject({
deps: [
{
currentDigest: 'b62d2a95c72f',
currentValue: 'nixpkgs-unstable',
datasource: 'git-refs',
depName: 'nixpkgs',
packageName: 'https://github.com/NixOS/nixpkgs',
rangeStrategy: 'update-lockfile',
},
],
});
Expand Down
84 changes: 63 additions & 21 deletions lib/modules/manager/nix/extract.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ const lockableHTTPTarballProtocol = regEx(
'^https://(?<domain>[^/]+)/(?<owner>[^/]+)/(?<repo>[^/]+)/archive/(?<rev>.+).tar.gz$',
);

const lockableChannelOriginalUrl = regEx(
'^https://nixos.org/channels/(?<channel>[^/]+)/nixexprs.tar.xz$',
);
const lockableChannelLockedUrl = regEx(
'^https://releases.nixos.org/nixpkgs/(?<channel>[^/-]+)-(?<release>[^/]+)pre[0-9]+.(?<ref>[^/]+)/nixexprs.tar.xz$',
);

export async function extractPackageFile(
content: string,
packageFile: string,
Expand All @@ -25,16 +32,19 @@ export async function extractPackageFile(

const deps: PackageDependency[] = [];

const match = nixpkgsRegex.exec(content);
if (match?.groups) {
const { ref } = match.groups;
deps.push({
depName: 'nixpkgs',
currentValue: ref,
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
versioning: nixpkgsVersioning,
});
const nixpkgsMatch = nixpkgsRegex.exec(content);
if (nixpkgsMatch?.groups) {
const { ref } = nixpkgsMatch.groups;
// only add when we matched a ref
if (ref !== undefined) {
deps.push({
depName: 'nixpkgs',
currentValue: ref,
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
versioning: nixpkgsVersioning,
});
}
Comment on lines +35 to +47
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const nixpkgsMatch = nixpkgsRegex.exec(content);
if (nixpkgsMatch?.groups) {
const { ref } = nixpkgsMatch.groups;
// only add when we matched a ref
if (ref !== undefined) {
deps.push({
depName: 'nixpkgs',
currentValue: ref,
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
versioning: nixpkgsVersioning,
});
}
const nixpkgsMatch = nixpkgsRegex.exec(content)?.groups;
// only add when we matched a ref
if (nixpkgsMatch?.ref) {
deps.push({
depName: 'nixpkgs',
currentValue: ref,
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
versioning: nixpkgsVersioning,
});

}

const flakeLockParsed = NixFlakeLock.safeParse(lockContents);
Expand Down Expand Up @@ -89,6 +99,14 @@ export async function extractPackageFile(
continue;
}

const isLockableTarball =
flakeOriginal.url && lockableChannelOriginalUrl.test(flakeOriginal.url);

// if no rev is being tracked, we cannot update this input
if (flakeLocked.rev === undefined && !isLockableTarball) {
continue;
}

switch (flakeLocked.type) {
case 'github':
deps.push({
Expand All @@ -97,6 +115,7 @@ export async function extractPackageFile(
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
packageName: `https://${flakeOriginal.host ?? 'github.com'}/${flakeOriginal.owner}/${flakeOriginal.repo}`,
rangeStrategy: 'update-lockfile',
});
break;
case 'gitlab':
Expand All @@ -106,6 +125,7 @@ export async function extractPackageFile(
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
packageName: `https://${flakeOriginal.host ?? 'gitlab.com'}/${decodeURIComponent(flakeOriginal.owner!)}/${flakeOriginal.repo}`,
rangeStrategy: 'update-lockfile',
});
break;
case 'git':
Expand All @@ -115,6 +135,7 @@ export async function extractPackageFile(
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
packageName: flakeOriginal.url,
rangeStrategy: 'update-lockfile',
});
break;
case 'sourcehut':
Expand All @@ -124,20 +145,41 @@ export async function extractPackageFile(
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
packageName: `https://${flakeOriginal.host ?? 'git.sr.ht'}/${flakeOriginal.owner}/${flakeOriginal.repo}`,
rangeStrategy: 'update-lockfile',
});
break;
case 'tarball':
deps.push({
depName,
currentValue: flakeLocked.ref,
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
// type tarball always contains this link
packageName: flakeOriginal.url!.replace(
lockableHTTPTarballProtocol,
'https://$<domain>/$<owner>/$<repo>',
),
});
if (isLockableTarball) {
const branch = flakeOriginal.url!.replace(
lockableChannelOriginalUrl,
'$<channel>',
);
const rev = flakeLocked.url!.replace(
lockableChannelLockedUrl,
'$<ref>',
);
deps.push({
depName,
currentValue: branch,
currentDigest: rev,
datasource: GitRefsDatasource.id,
packageName: 'https://github.com/NixOS/nixpkgs',
rangeStrategy: 'update-lockfile',
});
} else {
deps.push({
depName,
currentValue: flakeLocked.ref,
currentDigest: flakeLocked.rev,
datasource: GitRefsDatasource.id,
// type tarball always contains this link
packageName: flakeOriginal.url!.replace(
lockableHTTPTarballProtocol,
'https://$<domain>/$<owner>/$<repo>',
),
rangeStrategy: 'update-lockfile',
});
}
break;
// istanbul ignore next: just a safeguard
default:
Expand Down
Loading