Skip to content

Conversation

LebedevOleg1
Copy link
Collaborator

Changelog entry

[full-backup] Smoke test for full restore with simple schema change
...

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

[full-backup] Smoke test for full restore with simple schema change
...

Copy link

github-actions bot commented Oct 14, 2025

🟢 2025-10-14 12:30:15 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Oct 14, 2025

2025-10-14 11:43:52 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7f6d68c has started.
2025-10-14 11:44:00 UTC Artifacts will be uploaded here
2025-10-14 11:46:36 UTC ya make is running...
🟢 2025-10-14 11:49:11 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15 15 0 0 0 0

🟢 2025-10-14 11:49:14 UTC Build successful.
🟢 2025-10-14 11:49:24 UTC ydbd size 2.3 GiB changed* by +3.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 33eec4f merge: 7f6d68c diff diff %
ydbd size 2 423 187 176 Bytes 2 423 190 808 Bytes +3.5 KiB +0.000%
ydbd stripped size 515 834 952 Bytes 515 835 528 Bytes +576 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 14, 2025

2025-10-14 11:45:53 UTC Pre-commit check linux-x86_64-release-asan for 7f6d68c has started.
2025-10-14 11:46:07 UTC Artifacts will be uploaded here
2025-10-14 11:48:47 UTC ya make is running...
🟢 2025-10-14 11:50:17 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
3 3 0 0 0 0

🟢 2025-10-14 11:50:20 UTC Build successful.
🟢 2025-10-14 11:50:34 UTC ydbd size 3.8 GiB changed* by +6.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 33eec4f merge: 7f6d68c diff diff %
ydbd size 4 031 463 936 Bytes 4 031 470 096 Bytes +6.0 KiB +0.000%
ydbd stripped size 1 497 302 912 Bytes 1 497 304 640 Bytes +1.7 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@LebedevOleg1 LebedevOleg1 changed the title Test3 [full-backup] Smoke test for full restore with simple schema change Oct 14, 2025
@Enjection Enjection changed the title [full-backup] Smoke test for full restore with simple schema change [incr-backup] Smoke test for full restore with simple schema change Oct 14, 2025
@Enjection
Copy link
Member

Renamed to [incr-backup] as it is like tag for a feature in general, even if this particular test covers only full-backup functionality inside this bigger feature

Comment on lines 966 to 981
def _create_backup_collection(self, collection_src, tables: List[str]):
# create backup collection referencing given table full paths
table_entries = ",\n".join([f"TABLE `/Root/{t}`" for t in tables])
sql = f"""
CREATE BACKUP COLLECTION `{collection_src}`
( {table_entries} )
WITH ( STORAGE = 'cluster', INCREMENTAL_BACKUP_ENABLED = 'false' );
"""
res = self._execute_yql(sql)
stderr_out = ""
if getattr(res, 'std_err', None):
stderr_out += res.std_err.decode('utf-8', errors='ignore')
if getattr(res, 'std_out', None):
stderr_out += res.std_out.decode('utf-8', errors='ignore')
assert res.exit_code == 0, f"CREATE BACKUP COLLECTION failed: {stderr_out}"
self.wait_for_collection(collection_src, timeout_s=30)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to extract this to common base class to avoid repetition

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same applies to other "common" reusable methods as well)

res = self._execute_yql(cmd)
assert res.exit_code == 0, "Failed to apply GRANT in wave 2"

# capture state after wave 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this purpose usually word "stage" used

Comment on lines 1209 to 1212
try:
session.execute_scheme('DROP TABLE `/Root/extra_table_1`;')
except Exception:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should explicitly state expected behavior. For now we expect it to fail (even though it is not failing actually). So write test that actually fails on it and mark it as muted like it is done here https://github.com/ydb-platform/ydb/pull/26845/files) and create issue (e.g. Incremental backup must forbid removal of managed objects)

@LebedevOleg1 LebedevOleg1 self-assigned this Oct 14, 2025
Copy link

github-actions bot commented Oct 15, 2025

2025-10-15 13:05:04 UTC Pre-commit check linux-x86_64-release-asan for e844e24 has started.
2025-10-15 13:05:27 UTC Artifacts will be uploaded here
2025-10-15 13:08:45 UTC ya make is running...
🟢 2025-10-15 13:10:08 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
3 3 0 0 0 0

🟢 2025-10-15 13:10:11 UTC Build successful.
🟡 2025-10-15 13:10:26 UTC ydbd size 3.8 GiB changed* by +201.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6a97bc6 merge: e844e24 diff diff %
ydbd size 4 032 942 864 Bytes 4 033 148 680 Bytes +201.0 KiB +0.005%
ydbd stripped size 1 497 804 320 Bytes 1 497 883 744 Bytes +77.6 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 15, 2025

2025-10-15 13:05:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for e844e24 has started.
2025-10-15 13:05:50 UTC Artifacts will be uploaded here
2025-10-15 13:08:46 UTC ya make is running...
🟢 2025-10-15 13:11:24 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15 15 0 0 0 0

🟢 2025-10-15 13:11:27 UTC Build successful.
🟡 2025-10-15 13:11:38 UTC ydbd size 2.3 GiB changed* by +123.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6a97bc6 merge: e844e24 diff diff %
ydbd size 2 423 994 432 Bytes 2 424 120 744 Bytes +123.4 KiB +0.005%
ydbd stripped size 515 998 440 Bytes 516 026 664 Bytes +27.6 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

# remove some tables from step5: drop extra_table_1
try:
session.execute_scheme('DROP TABLE `/Root/extra_table_1`;')
except Exception:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST fail for t1 and t2 until they are under backup collection. We must allow dropping tables only when backup collection referencing this tables dropped. (Or maybe we need to reconsider this semantics)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, make assert here and create issue, please

Copy link

github-actions bot commented Oct 15, 2025

2025-10-15 19:30:47 UTC Pre-commit check linux-x86_64-release-asan for b8d4c68 has started.
2025-10-15 19:31:31 UTC Artifacts will be uploaded here
2025-10-15 19:34:37 UTC ya make is running...
🟢 2025-10-15 19:36:07 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
3 3 0 0 0 0

🟢 2025-10-15 19:36:10 UTC Build successful.
🟢 2025-10-15 19:36:24 UTC ydbd size 3.8 GiB changed* by +8.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 37977bb merge: b8d4c68 diff diff %
ydbd size 4 033 053 144 Bytes 4 033 062 040 Bytes +8.7 KiB +0.000%
ydbd stripped size 1 497 928 672 Bytes 1 497 932 480 Bytes +3.7 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 15, 2025

2025-10-15 19:32:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for b8d4c68 has started.
2025-10-15 19:32:42 UTC Artifacts will be uploaded here
2025-10-15 19:35:14 UTC ya make is running...
🟢 2025-10-15 19:37:47 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15 15 0 0 0 0

🟢 2025-10-15 19:37:50 UTC Build successful.
🟢 2025-10-15 19:38:00 UTC ydbd size 2.3 GiB changed* by +5.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 37977bb merge: b8d4c68 diff diff %
ydbd size 2 424 051 952 Bytes 2 424 057 632 Bytes +5.5 KiB +0.000%
ydbd stripped size 516 042 088 Bytes 516 043 304 Bytes +1.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants