Improve gitlab hudge repo import #420
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix1: Regex are nos supprted as `Allowed Branches`. Supported input are either a 'comma-separated list', or a 'regex' or '*' Fix2: Because Gitlab REST v4 API returns are paginated, we have to retrieve only interested branches
When your Gatlab repo is old and hudge, may be you should not import all your repo. Commits messages may respect the correct layout and do not refer any mantis ticket number. This feature can also avoid erors during very large imports. When your report is very hudge, reimport all commits could be very timeconsuming, and the result is so huge that php engine can raise a memory error. fix "deprecated dynamic parrameter" creation.
This was referenced Oct 1, 2025
|
Thanks for your contribution. I will have a look as time allows. |
|
This PR is mixing several things
Problem: you are making changes to MantisSourceGitBasePlugin class, which is used by all Git-based plugins, but only implementing for GitLab which is likely to introduce regressions for the others (I did not actually test). I would suggest you split your changes into 3 separate PRs so these can be reviewed and tested separately. If you want to create an Issue to describe the feature and reference it in the PR that's fine (but not strictly needed). For 2, you can refer to the existing #389. Thanks for your understanding. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi,
Here is my journey, with the
Gitlabplugin.First I have tried to import everithing. Because my repo is very hudge with lots of commits across to many old branches,
the import process raise a
php memory error:When I check into the database, I can see that commits have been imported.
But it was only old commits.
I have tried to configure the Allowed branches fields with a regex to limit the number of branches to look up but it failed.
Even after fix this issue (see commit 7528e98), I still have my import issue.
The root cause of my import issue is on gitlab, the branches API return only the 20 first branches of the projet.
Results are paginated. We have to iterate over this API and play with
per_pageandpageparameters, ex :https://gitlab.mycompagny.com/api/v4/projects/id/repository/branches?per_page=100&page=1"
So in commit c005c2f, I propose several improvments :
This date allows to limit the number of commit to import.
On old projet, commits messages may not respects the pattern that allow link a commit with an issue. So it is no use to import then.
I hope this proposal will be intégrated in next versions.
Regards,
Nicolas