Some assets lost with a link pointing to 404 after migration from 1.x

[pansila]

Describe the bug
Some user assets seem to be lost, the link of them is pointing to 404. But the new pictures uploaded after migration shows correctly. BTW, I'm using sqlite database and local folder storage.

To Reproduce
Steps to reproduce the behavior:

1. following migration guide from 1.x to 2.x
2. Check the assets in the page, the link color is red if it's broken.

Expected behavior
All user assets should be accessible after migration.

Screenshots

Host Info (please complete the following information):

• OS: Windows 7 64bit
• Wiki.js version: 2.0.1
• Database engine: Sqlite

Additional context
Add any other context about the problem here.

[NGPixel]

Which migration option did you use? Git or Local import?

[pansila]

@NGPixel Yes, It's Local import.

[pansila]

@NGPixel After some debug, I found it's a hash mismatch problem, the hash in the sqlite database is different from what is calculated when the asset is requested. There might be a bug in the migration from 1.x to 2.0 when calculating the asset hashes.

[pansila]

It's the path seperating character. on Linux, it's forward slash '/' where on Windows it's back slash '\'. In the server core, it seems that paths are unified to using forward slash, while in the migration tools it's using OS-dependent path seperating character. That makes the hash mismatch.

[pansila]

Here is the python script to correct the asset hash values in the sqlite database if you don't want to migrate the wiki again after the bug is fixed.

import sqlite3
import hashlib

DBFile = 'd:\\path\\to\\wiki-database.sqlite'


def getPath(folder, folder_list):
  if folder == 'uploads':
    return ['uploads']
  for f in folder_list:
    if folder == f[1]:
      if f[2] == 1:
        return ['uploads', f[1]]
      return getPath(f[3], folder_list) + [f[1]]
  else:
    return []

conn = sqlite3.connect(DBFile)
c = conn.cursor()

c.execute("SELECT a1.id, a1.name, a2.id, a2.name FROM assetFolders AS a1 inner join assetFolders AS a2 on a1.parentId == a2.id")

folder_list = []
while True:
  ret = c.fetchone()
  if not ret:
    break
  folder_list.append(ret)

c.execute("SELECT name, filename, hash, assets.id FROM assets inner join assetFolders on assets.folderid == assetFolders.id")

new_hash = []

while True:
  ret = c.fetchone()
  if not ret:
    break
  filePath = getPath(ret[0], folder_list) + [ret[1]]
  hash1 = hashlib.sha1('/'.join(filePath).encode('utf8')).hexdigest()
  hash2 = hashlib.sha1('\\'.join(filePath).encode('utf8')).hexdigest()

  if ret[2] == hash2:
    new_hash.append([ret[3], hash1])

for item in new_hash:
  c.execute(f'UPDATE assets set hash = "{item[1]}" where id == {item[0]}')
conn.commit()

conn.close()