br: error if the log restore has no full backup schema or id maps

[Leavrth]

What problem does this PR solve?

Issue Number: close #54418

Problem Summary:
Performing full restore and log restore separately will be a disaster

What changed and how does it work?

output error if the log restore has no full backup schema or id maps

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Prevent the cluster from performing a PITR recovery for the first time without specifying --full-backup-storage.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from leavrth, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• br/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tiprow]

Hi @Leavrth. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[codecov]

Codecov Report

Attention: Patch coverage is 14.28571% with 6 lines in your changes missing coverage. Please review.

Project coverage is 56.9162%. Comparing base (91a165b) to head (9da44d3).
Report is 42 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #54421         +/-   ##
=================================================
- Coverage   72.8282%   56.9162%   -15.9121%     
=================================================
  Files          1533       1649        +116     
  Lines        436067     606839     +170772     
=================================================
+ Hits         317580     345390      +27810     
- Misses        98835     237700     +138865     
- Partials      19652      23749       +4097     

Flag	Coverage Δ
integration	37.8601% <14.2857%> (?)
unit	71.8290% <0.0000%> (+0.0055%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	52.9656% <ø> (ø)
parser	∅ <ø> (∅)
br	62.7321% <14.2857%> (+16.6494%)	⬆️

[ti-chi-bot]

@Leavrth: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/check_dev_2	9da44d3	link	true	/test check-dev2

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[kennytm]

Hold on, will this make Full -> Incremental -> PITR (supply --start-ts only) impossible to run (assume incremental has no DDLs, and ignore the RewriteTS issue at the moment)

[Leavrth]

Hold on, will this make Full -> Incremental -> PITR (supply --start-ts only) impossible to run (assume incremental has no DDLs, and ignore the RewriteTS issue at the moment)

Yes, this PR prohibits the behavior even if there is no rewrite ts problem. The issue #54418 provides an example.

[kennytm]

@Leavrth is the --start-ts flag still usable?

(anyway i'm tagging release-note this is behavior-changing.)

[Leavrth]

@Leavrth is the --start-ts flag still usable?

(anyway i'm tagging release-note this is behavior-changing.)

Yes. Currently log restore will upload a file to the external storage, which contains id-maps from the upstream table/db ids to the pre-allocated downstream table/db ids at restored-ts.
This path of id-maps is:

tidb/br/pkg/metautil/metafile.go

Line 62 in 7a0611f

return fmt.Sprintf("%s/pitr_id_map.cluster_id:%d.restored_ts:%d", "pitr_id_maps", clusterID, restoreTS)

With the id-maps,

1. log restore can retry with checkpoint if it failed.
2. if log restore with --restored-ts T1 is finished. The next log restore with start-ts T1 but without full-backup-storage will read the id-maps pitr_id_maps/pitr_id_map.cluster_id:{cluster-id}.restored_ts:{start-ts} instead of reading the tableinfos in backupmeta.
   

   tidb/br/pkg/restore/log_client/client.go

   Lines 674 to 693 in 7a0611f

   	// a new task, but without full snapshot restore, tries to load
   	// schemas map whose `restore-ts`` is the task's `start-ts`.
   	if len(dbMaps) <= 0 && !cfg.HasFullRestore {
   	log.Info("try to load pitr id maps of the previous task", zap.Uint64("start-ts", rc.startTS))
   	needConstructIdMap = true
   	dbMaps, err = rc.initSchemasMap(ctx, rc.GetClusterID(ctx), rc.startTS)
   	if err != nil {
   	return nil, errors.Trace(err)
   	}
   	info := rc.dom.InfoSchema()
   	shcemas := info.AllSchemaNames()
   	for _, schema := range shcemas {
   	for _, table := range info.SchemaTables(schema) {
   	tableInfo := table.Meta()
   	if tableInfo.TiFlashReplica != nil && tableInfo.TiFlashReplica.Count > 0 {
   	return nil, errors.Errorf("exist table(s) have tiflash replica, please remove it before restore")
   	}
   	}
   	}
   	}

[kennytm]

it seems id-map was introduced in v7.1.0, so --start-ts requires an archive made from v7.1 at least I guess.