Add feature to merge .bib files into current bib #13320

raquelgraos · 2025-06-13T11:28:16Z

This PR introduces a feature that allows for users to merge .bib files in a chosen directory into their current bib.
If an imported entry is equal to an existent one, it is silently ignored. If it is a duplicate (according to JabRef's duplication algorithm) or has the same citation key, it can either be silently ignored or the entries are merged, using JabRef's merge dialog (users can configure their preference in the Preferences menu in the "Merge other bib files into current bib" tab).
Users can undo/redo this command.

Co-authored-by: Guilherme Ribeiro Pereira guilherme.ribeiro.pereira@tecnico.ulisboa.pt

Steps to test

Open (or create) a library
Go to Files > Merge other libraries into current library...
Select the directory that contains the .bib files to be merged (these .bib files should have some equal/duplicate/same citation key entries compared to the ones in the library described in step 1)
Repeat this process changing the duplicate/same citation key handling preferences in File > Preferences > Merge other .bib files into current .bib

Feature_Demonstration.webm

Automatic tests were also created for this PR.

Mandatory checks

I own the copyright of the code submitted and I license it under the MIT license
Change in CHANGELOG.md described in a way that is understandable for the average user (if change is visible to the user)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (if change is visible to the user)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

trag-bot · 2025-06-13T11:30:54Z

.../java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibPreferences.java

+        this.shouldMergeDuplicateEntries.set(shouldMergeDuplicateEntries);
+    }
+
+    public boolean getShouldMergeSameKeyEntries() {


Method name violates Java bean naming conventions. Boolean getter methods should start with 'is' rather than 'get' for better readability and convention compliance.

in this case i would opt to leave the whole get or is away. we do that in other preferences too.

Hi! Thanks for the feedback, we fixed that

calixtus · 2025-06-14T13:44:42Z

...a/org/jabref/gui/preferences/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibTab.java

Isn't this a bit of overkill, for two checkboxes a new preferences tab and preferences object?
I think this can be merged into another tab and another prefs object.
Think of the user. Where would the user look for these preferences options?

If there are similar preferences options in other tabs and preferences objects, your welcome to propose a new plan and a new prefs tab to collect all these similar option.

And the name is just horrible.

Hi! Thank you for the feedback!
We'll have another look at the Preferences menu to see where they fit better.
Regarding the name, do you have any suggestions?

This feature allows for users to merge .bib files in a chosen directory into their current bib. If an imported entry is equal to an existent one, it is silently ignored. If it is a duplicate or has the same citation key, it can either be silently ignored or the entries are merged (users can configure their preference in the Preferences menu in the "Merge other bib files into current bib" tab). Users can also undo/redo this command. Fixes JabRef#12290 Co-authored-by: Guilherme Ribeiro Pereira <guilherme.ribeiro.pereira@tecnico.ulisboa.pt>

- Replace 'bib files' to standardized 'BibTeX' in StandardActions - Remove redundant Javadoc in MergeBibFilesIntoCurrentBibAction - Rename boolean getters - Delete obsolete test comments - Correct Changelog issue link

- Replace 'bib' to standardized 'BibTeX' in StandardActions - Delete obsolete test comments - Corrected MergeBibFilesIntoCurrentBib string in JabRef_en.properties

- Replace 'bib' to standardized 'BibTeX' in all strings related to the merge libraries action - Corrected jabRef_en.properties according to the changes above

calixtus · 2025-06-15T16:27:29Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        } catch (IOException e) {
+            LOGGER.error("Error finding .bib files in '{}'", directory.getFileName(), e);
+        }


I think the exception is thrown if the Files.find method cannot open or read the path, so I think there should be check for if the path is valid or not to have correct and helpful feedback to the user (including a message).

calixtus · 2025-06-15T16:27:42Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        BibEntryTypesManager entryTypesManager = new BibEntryTypesManager();
+        DuplicateCheck dupCheck = new DuplicateCheck(entryTypesManager);
+
+        for (Path p : getAllBibFiles(directory, databasePath.orElseGet(() -> Path.of("")))) {


Please avoid abbreviations

calixtus · 2025-06-15T16:27:54Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+
+            for (BibEntry toMergeEntry : result.getDatabase().getEntries()) {
+                boolean validNewEntry = true;
+                for (BibEntry e : database.getEntries()) {


Please avoid abbreviations

calixtus · 2025-06-15T16:28:20Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+                }
+            }
+        }
+        NamedCompound ce = new NamedCompound(Localization.lang("Merge BibTeX files into current library"));


Please avoid abbreviations

calixtus · 2025-06-15T16:31:00Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        BibDatabase database = context.getDatabase();
+        Optional<Path> databasePath = context.getDatabasePath();
+
+        BibEntryTypesManager entryTypesManager = new BibEntryTypesManager();


calixtus · 2025-06-15T16:32:48Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        return List.of();
+    }
+
+    public ParserResult loadDatabase(Path file) {


Why don't you reuse existing methods? Looks like code duplication to me.
If not, put some comment on it.

calixtus · 2025-06-15T16:38:05Z

...rc/test/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibTest.java

+    void setUp() throws IOException {
+        MockitoAnnotations.openMocks(this);
+
+        FileSystem fs = Jimfs.newFileSystem(Configuration.unix());


Why aren't you using TempDir? Are there features neccessary provided by JimFS?
https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/io/TempDir.html
We already use it for testing in other classes.

Please avoid introducing new dependencies if not really needed.

calixtus · 2025-06-15T16:41:44Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+                            selectedEntries = new ArrayList<>();
+                            selectedEntries.add(toMergeEntry);
+                            selectedEntries.add(e);
+                            stateManager.setSelectedEntries(selectedEntries);


changes to the ui should only happen at the very end of the action.

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

GuilhermeRibeiroPereira · 2025-06-16T16:07:26Z

Thank you for the feedback! We will fix the problems mentioned

- Tests now use JUnit TempDir

…references tab - Removed abbreviations in MergeBibFilesIntoCurrentBibAction - Added a check for the chosen directory's path validity - Merges are performed at the end of the action - Refactored MergeBibFilesIntoCurrentBib method - Preferences related to the action moved to the General tab

- Applied OpenRewrite automated code improvements - Removed unnecessary test field in duplicate tests

calixtus · 2025-06-20T17:49:53Z

...rc/test/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibTest.java

@@ -250,14 +250,12 @@ public void sameCitationKeyNoMergePreferenceTest() {
    @Test
    public void duplicateMergeTest() {
        BibEntry currentEntry = new BibEntry(StandardEntryType.Article)
-                .withCitationKey("DIFFERENTCITATIONKEY")
+                .withCitationKey("DIFFERENT CITATION KEY")


I don't think bibtex or biblatex allow multiple words as citation keys. The documentation does not say anything about this, but there are hints. I was looking into the source of biber but was not able to quickly locate the parser for citation keys (https://github.com/plk/biber/blob/dev/lib/Biber.pm).
But anyways, please remove whitespaces from the citation key, just to make sure.

Its possible with BibLaTeX: https://tex.stackexchange.com/a/37646/9075

However, it is very seldomly used and we have no proper UI support in JabRef for it.

Neither spaces nor commans are allowed in the key references: https://tex.stackexchange.com/a/271520/9075

- Fix error message - Remove whitespaces from citation keys

GuilhermeRibeiroPereira · 2025-06-28T16:35:59Z

Hi @calixtus, when you have some time, could you check out our changes?
Thanks!

subhramit · 2025-06-30T19:20:20Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        return List.of();
+    }
+
+    private boolean checkPathValidity(Path directory) {


Naming nitpick:
"checkPathValidity" sounds like a function that doesn't return anything.
A better name would be isValidPath so that it can be used in conditionals as if(isValidPath(...)) rather than if(checkPathValidity(...))

Already changed it! Thanks!

jabgui/src/main/java/org/jabref/gui/preferences/JabRefGuiPreferences.java

…s listeners

trag-bot · 2025-07-07T13:11:11Z

jabgui/src/main/java/org/jabref/gui/frame/MainMenu.java

@@ -181,6 +182,10 @@ private void createMenu() {

                new SeparatorMenuItem(),

+                factory.createMenuItem(StandardActions.MERGE_BIBTEX_FILES_INTO_CURRENT_LIBRARY, new MergeBibFilesIntoCurrentBibAction(dialogService, preferences, stateManager, undoManager, fileUpdateMonitor, entryTypesManager)),


The term 'BIBTEX' in the constant name doesn't follow the convention of using 'Bibtex' in variable/constant names as specified in special instructions.

koppor · 2025-07-07T19:30:11Z

We need to test on our libraries to see if this matches our expectations 😅

trag-bot · 2025-07-07T19:32:27Z

@trag-bot didn't find any issues in the code! ✅✨

koppor · 2025-07-07T20:35:24Z

Thank you for working on this and taking the challenge. The issue was written in 2018 and things changed meanwhile.

While reviewing your code, I found org.jabref.logic.bibtex.comparator.BibDatabaseDiff#compare

Task:

Replace your whole process entry logic etc by calling this compare and iterating thought org.jabref.logic.bibtex.comparator.BibDatabaseDiff#getEntryDifferences.

first parameter: current library, second parameter: lbrary to merge
if originalEntry is null - just copy into the library
if originalEntry and newEntry are not null: show merge dialog
if newEntry is null - ignore (entry exists in original library only)

Remove all your preferences. These are more disturbing than useful currently.

Follow-up

Things not adressed in the PR

Copy fiiles (if they are not reachable from the current library) --> needs more refinement
merge of groups and other meta data; strings; and preamble

Side Comments

Preferences

Title should be: "Merging of libraries" to know that it is the fucntionality (the other title "Merge libraries functionality" is too long/confusing)

It is unclear, what happens if the options are not selected.

I see options

Imported as sseperate entry
Ignored

I think, you mean "ignore". This is not clear to the user. - We could adress this by a tooltipo - or by a readio button group. However, I think, this is too much effort for the gain.

When checked, you open a merge dialog for these.

koppor

Longer comment above

I think, my comments are obsolete with the rewrite I asked for at the comment above.

koppor · 2025-07-07T20:16:04Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+        sameKeyPairsToMerge.clear();
+        duplicatePairsToMerge.clear();
+
+        for (Path path : getAllBibFiles(directory, databasePath.orElseGet(() -> Path.of("")))) {


Why is the current library ("databasePath") included? I think, one wants to merge into the current one?

koppor · 2025-07-07T20:16:36Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+            try {
+                result = OpenDatabase.loadDatabase(path, preferences.getImportFormatPreferences(), fileUpdateMonitor);
+            } catch (IOException e) {
+                LOGGER.error("Could not load file '{}': {}", path, e.getMessage(), e);


It should be a dialogService.notify(...) in additon. Look at JabRef_en.properties for a nice string to re-use.

koppor · 2025-07-07T20:18:48Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+            } else if (entry.getCitationKey().equals(existingEntry.getCitationKey())) {
+                if (shouldMergeSameKeyEntries) {
+                    sameKeyPairsToMerge.add(List.of(entry, existingEntry));
+                }
+                return;


No - check for org.jabref.model.database.BibDatabase#getEntryByCitationKey first.

Imagine a library with 10k entries - using a hashmap is faster than iterating through all entries.

koppor · 2025-07-07T20:25:25Z

.../main/java/org/jabref/gui/mergebibfilesintocurrentbib/MergeBibFilesIntoCurrentBibAction.java

+                (path, _) -> path.getFileName().toString().endsWith(".bib") &&
+                        !path.equals(databasePath)
+        )) {
+            return stream.collect(Collectors.toList());


Suggested change

return stream.collect(Collectors.toList());

return stream.toList();

trag-bot bot reviewed Jun 13, 2025

View reviewed changes

calixtus reviewed Jun 14, 2025

View reviewed changes

raquelgraos and others added 2 commits June 15, 2025 11:47

fix: cleanup MergeBibFilesIntoCurrentBib

e415652

- Replace 'bib files' to standardized 'BibTeX' in StandardActions - Remove redundant Javadoc in MergeBibFilesIntoCurrentBibAction - Rename boolean getters - Delete obsolete test comments - Correct Changelog issue link

GuilhermeRibeiroPereira force-pushed the fix-for-issue-12290 branch from 43bcbb0 to e415652 Compare June 15, 2025 12:49

GuilhermeRibeiroPereira and others added 2 commits June 15, 2025 15:10

fix: cleanup MergeBibFilesIntoCurrentBib

984133d

- Replace 'bib' to standardized 'BibTeX' in StandardActions - Delete obsolete test comments - Corrected MergeBibFilesIntoCurrentBib string in JabRef_en.properties

fix: cleanup MergeBibFilesIntoCurrentBib

4987209

- Replace 'bib' to standardized 'BibTeX' in all strings related to the merge libraries action - Corrected jabRef_en.properties according to the changes above

calixtus requested changes Jun 15, 2025

View reviewed changes

GuilhermeRibeiroPereira and others added 5 commits June 16, 2025 19:16

fix: removed Jimfs dependency for testing

81cd997

- Tests now use JUnit TempDir

Merge remote-tracking branch 'upstream/main' into fix-for-issue-12290

e7736a4

Merge remote-tracking branch 'upstream/main' into fix-for-issue-12290

ac0b688

fix: apply OpenRewrite autofixes and clean up duplicate tests

e15d40d

- Applied OpenRewrite automated code improvements - Removed unnecessary test field in duplicate tests

raquelgraos requested a review from calixtus June 20, 2025 14:13

calixtus reviewed Jun 20, 2025

View reviewed changes

fix: clean up tests

a358457

- Fix error message - Remove whitespaces from citation keys

raquelgraos requested a review from calixtus June 21, 2025 19:24

subhramit reviewed Jun 30, 2025

View reviewed changes

jabgui/src/main/java/org/jabref/gui/preferences/JabRefGuiPreferences.java Outdated Show resolved Hide resolved

raquelgraos and others added 2 commits June 30, 2025 22:46

Refactor:renamed path validity method & put placeholders in propertie…

e892ec3

…s listeners

Merge remote-tracking branch 'upstream/main' into fix-for-issue-12290

16c4365

trag-bot bot reviewed Jul 7, 2025

View reviewed changes

Merge branch 'main' into fix-for-issue-12290

9b45c43

koppor requested changes Jul 7, 2025

View reviewed changes

		@@ -181,6 +182,10 @@ private void createMenu() {

		new SeparatorMenuItem(),

		factory.createMenuItem(StandardActions.MERGE_BIBTEX_FILES_INTO_CURRENT_LIBRARY, new MergeBibFilesIntoCurrentBibAction(dialogService, preferences, stateManager, undoManager, fileUpdateMonitor, entryTypesManager)),

	return stream.collect(Collectors.toList());
	return stream.toList();

Uh oh!

Add feature to merge .bib files into current bib #13320

Are you sure you want to change the base?

Add feature to merge .bib files into current bib #13320

Uh oh!

Conversation

raquelgraos commented Jun 13, 2025

Steps to test

Mandatory checks

Uh oh!

trag-bot bot Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GuilhermeRibeiroPereira commented Jun 16, 2025

Uh oh!

calixtus Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GuilhermeRibeiroPereira commented Jun 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trag-bot bot Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

koppor commented Jul 7, 2025

Uh oh!

trag-bot bot commented Jul 7, 2025

Uh oh!

koppor commented Jul 7, 2025

Follow-up

Side Comments

Preferences

Uh oh!

koppor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

calixtus Jun 20, 2025 •

edited

Loading