Skip to content

Show fetch exception at citation relation #13549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+117 −82
Open

Show fetch exception at citation relation #13549

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
563b8ff
Show fetch exception at citation relation
koppor Jul 15, 2025
5165288
Add link
koppor Jul 15, 2025
7068b25
Fix typo
koppor Jul 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
- We added the field `monthfiled` to the default list of fields to resolve BibTeX-Strings for [#13375](https://github.com/JabRef/jabref/issues/13375)
- We added a new ID based fetcher for [EuropePMC](https://europepmc.org/). [#13389](https://github.com/JabRef/jabref/pull/13389)
- We added an initial [cite as you write](https://retorque.re/zotero-better-bibtex/citing/cayw/) endpoint. [#13187](https://github.com/JabRef/jabref/issues/13187)
- In case no citation relation information can be fetched, we show the data providers reason. [#13549](https://github.com/JabRef/jabref/pull/13549)

### Changed

Expand Down
2 changes: 1 addition & 1 deletion jabgui/src/main/java/org/jabref/gui/collab/groupchange/GroupChangeDetailsView.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
public final class GroupChangeDetailsView extends DatabaseChangeDetailsView {

public GroupChangeDetailsView(GroupChange groupChange) {
String labelValue = "";
String labelValue;
if (groupChange.getGroupDiff().getNewGroupRoot() == null) {
labelValue = groupChange.getName() + '.';
} else {
Expand Down
82 changes: 46 additions & 36 deletions ...ui/src/main/java/org/jabref/gui/entryeditor/citationrelationtab/CitationRelationsTab.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,22 +201,22 @@ private SplitPane getPaneAndStartSearch(BibEntry entry) {

refreshCitingButton.setOnMouseClicked(_ -> {
searchForRelations(
entry,
citingListView,
entry,
citingListView,
abortCitingButton,
refreshCitingButton,
CitationFetcher.SearchType.CITES,
importCitingButton,
refreshCitingButton,
CitationFetcher.SearchType.CITES,
importCitingButton,
citingProgress);
});

refreshCitedByButton.setOnMouseClicked(_ -> searchForRelations(
entry,
citedByListView,
entry,
citedByListView,
abortCitedButton,
refreshCitedByButton,
CitationFetcher.SearchType.CITED_BY,
importCitedByButton,
refreshCitedByButton,
CitationFetcher.SearchType.CITED_BY,
importCitedByButton,
citedByProgress));

// Create SplitPane to hold all nodes above
Expand All @@ -225,21 +225,21 @@ private SplitPane getPaneAndStartSearch(BibEntry entry) {
styleFetchedListView(citingListView);

searchForRelations(
entry,
citingListView,
abortCitingButton,
entry,
citingListView,
abortCitingButton,
refreshCitingButton,
CitationFetcher.SearchType.CITES,
importCitingButton,
CitationFetcher.SearchType.CITES,
importCitingButton,
citingProgress);

searchForRelations(
entry,
citedByListView,
abortCitedButton,
entry,
citedByListView,
abortCitedButton,
refreshCitedByButton,
CitationFetcher.SearchType.CITED_BY,
importCitedByButton,
CitationFetcher.SearchType.CITED_BY,
importCitedByButton,
citedByProgress);

return container;
Expand Down Expand Up @@ -444,14 +444,13 @@ private void searchForRelations(BibEntry entry, CheckListView<CitationRelationIt
return;
}

ObservableList<CitationRelationItem> observableList = FXCollections.observableArrayList();

listView.setItems(observableList);
ObservableList<CitationRelationItem> resultList = FXCollections.observableArrayList();
listView.setItems(resultList);

// TODO: It should not be possible to cancel a search task that is already running for same tab
if (citingTask != null && !citingTask.isCancelled() && searchType == CitationFetcher.SearchType.CITES) {
if (searchType == CitationFetcher.SearchType.CITES && citingTask != null && !citingTask.isCancelled()) {
citingTask.cancel();
} else if (citedByTask != null && !citedByTask.isCancelled() && searchType == CitationFetcher.SearchType.CITED_BY) {
} else if (searchType == CitationFetcher.SearchType.CITED_BY && citedByTask != null && !citedByTask.isCancelled()) {
citedByTask.cancel();
}

Expand All @@ -468,13 +467,22 @@ private void searchForRelations(BibEntry entry, CheckListView<CitationRelationIt
importButton,
progress,
fetchedList,
observableList
resultList
))
.onFailure(exception -> {
LOGGER.error("Error while fetching citing Articles", exception);
LOGGER.error("Error while fetching {} papers",
searchType == CitationFetcher.SearchType.CITES ? "cited" : "citing",
exception);
hideNodes(abortButton, progress, importButton);
listView.setPlaceholder(new Label(Localization.lang("Error while fetching citing entries: %0",
exception.getMessage())));
String labelText;
if (searchType == CitationFetcher.SearchType.CITES) {
labelText = Localization.lang("Error while fetching cited entries: %0", exception.getMessage());
} else {
labelText = Localization.lang("Error while fetching citing entries: %0", exception.getMessage());
}
Label placeholder = new Label(labelText);
placeholder.setWrapText(true);
listView.setPlaceholder(placeholder);
refreshButton.setVisible(true);
dialogService.notify(exception.getMessage());
})
Expand All @@ -490,13 +498,13 @@ private BackgroundTask<List<BibEntry>> createBackgroundTask(
return switch (searchType) {
case CitationFetcher.SearchType.CITES -> {
citingTask = BackgroundTask.wrap(
() -> this.searchCitationsRelationsService.searchReferences(entry)
() -> this.searchCitationsRelationsService.searchCites(entry)
);
yield citingTask;
}
case CitationFetcher.SearchType.CITED_BY -> {
citedByTask = BackgroundTask.wrap(
() -> this.searchCitationsRelationsService.searchCitations(entry)
() -> this.searchCitationsRelationsService.searchCitedBy(entry)
);
yield citedByTask;
}
Expand All @@ -510,6 +518,8 @@ private void onSearchForRelationsSucceed(BibEntry entry, CheckListView<CitationR
ObservableList<CitationRelationItem> observableList) {
hideNodes(abortButton, progress);

// TODO: This could be a wrong database, because the user might have switched to another library
// If we were on fixing this, we would need to a) associate a BibEntry with a database or b) pass the database at "bindToEntry"
BibDatabase database = stateManager.getActiveDatabase().map(BibDatabaseContext::getDatabase)
.orElse(new BibDatabase());
observableList.setAll(
Expand All @@ -523,15 +533,15 @@ private void onSearchForRelationsSucceed(BibEntry entry, CheckListView<CitationR
.toList()
);

if (!observableList.isEmpty()) {
listView.refresh();
} else {
if (observableList.isEmpty()) {
Label placeholder = new Label(Localization.lang("No articles found"));
listView.setPlaceholder(placeholder);
} else {
listView.refresh();
}
BooleanBinding booleanBind = Bindings.isEmpty(listView.getCheckModel().getCheckedItems());
importButton.disableProperty().bind(booleanBind);
importButton.setOnMouseClicked(event -> importEntries(listView.getCheckModel().getCheckedItems(), searchType, entry));
importButton.setOnMouseClicked(_ -> importEntries(listView.getCheckModel().getCheckedItems(), searchType, entry));
showNodes(refreshButton, importButton);
}

Expand All @@ -540,7 +550,7 @@ private void prepareToSearchForRelations(Button abortButton, Button refreshButto
showNodes(abortButton, progress);
hideNodes(refreshButton, importButton);

abortButton.setOnAction(event -> {
abortButton.setOnAction(_ -> {
hideNodes(abortButton, progress, importButton);
showNodes(refreshButton);
task.cancel();
Expand Down
2 changes: 2 additions & 0 deletions jablib/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,8 @@ dependencies {

implementation("com.fasterxml:aalto-xml")

implementation("org.hisp.dhis:json-tree")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why another json library?

Copy link
Member

@Siedlerchr Siedlerchr Jul 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For optionals use kongs unirest json

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why another json library?

Because I know the author personally. And it is fast

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For optionals use kongs unirest json

Show me. 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I know the author personally. And it is fast
Argument by authority is not a valid reason

see: eg.g
JSONObject result = item.getJSONObject("resultList").getJSONArray("result").getJSONObject(0);

jabref/jablib/src/main/java/org/jabref/logic/importer/fetcher/EuropePmcFetcher.java

Line 117 in e29044c

Optional.ofNullable(result.optString("pubModel")).ifPresent(pubModel -> entry.setField(StandardField.HOWPUBLISHED, pubModel));


implementation("org.postgresql:postgresql")

antlr("org.antlr:antlr4")
Expand Down
1 change: 1 addition & 0 deletions jablib/src/main/java/module-info.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@
requires com.fasterxml.jackson.databind;
requires com.fasterxml.jackson.dataformat.yaml;
requires com.fasterxml.jackson.datatype.jsr310;
requires org.hisp.dhis.jsontree;
// endregion

// region HTTP clients
Expand Down
37 changes: 14 additions & 23 deletions jablib/src/main/java/org/jabref/logic/citation/SearchCitationsRelationsService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,46 +39,37 @@ public SearchCitationsRelationsService(ImporterPreferences importerPreferences,
);
}

/**
* @implNote Typically, this would be a Shim in JavaFX
*/
@VisibleForTesting
public SearchCitationsRelationsService(CitationFetcher citationFetcher,
SearchCitationsRelationsService(CitationFetcher citationFetcher,
BibEntryCitationsAndReferencesRepository repository
) {
this.citationFetcher = citationFetcher;
this.relationsRepository = repository;
}

public List<BibEntry> searchReferences(BibEntry referenced) {
boolean isFetchingAllowed = relationsRepository.isReferencesUpdatable(referenced)
|| !relationsRepository.containsReferences(referenced);
public List<BibEntry> searchCites(BibEntry referencing) throws FetcherException {
boolean isFetchingAllowed =
!relationsRepository.containsReferences(referencing) ||
relationsRepository.isReferencesUpdatable(referencing);
if (isFetchingAllowed) {
try {
List<BibEntry> referencedBy = citationFetcher.searchCiting(referenced);
relationsRepository.insertReferences(referenced, referencedBy);
} catch (FetcherException e) {
LOGGER.error("Error while fetching references for entry {}", referenced.getTitle(), e);
}
List<BibEntry> referencedBy = citationFetcher.searchCiting(referencing);
relationsRepository.insertReferences(referencing, referencedBy);
}
return relationsRepository.readReferences(referenced);
return relationsRepository.readReferences(referencing);
}

/**
* If the store was empty and nothing was fetch in any case (empty fetch, or error) then yes => empty list
* If the store was not empty and nothing was fetched after a successful fetch => the store will be erased and the returned collection will be empty
* If the store was not empty and an error occurs while fetching => will return the content of the store
*/
public List<BibEntry> searchCitations(BibEntry cited) {
boolean isFetchingAllowed = relationsRepository.isCitationsUpdatable(cited)
|| !relationsRepository.containsCitations(cited);
public List<BibEntry> searchCitedBy(BibEntry cited) throws FetcherException {
boolean isFetchingAllowed =
!relationsRepository.containsCitations(cited) ||
relationsRepository.isCitationsUpdatable(cited);
if (isFetchingAllowed) {
try {
List<BibEntry> citedBy = citationFetcher.searchCitedBy(cited);
relationsRepository.insertCitations(cited, citedBy);
} catch (FetcherException e) {
LOGGER.error("Error while fetching citations for entry {}", cited.getTitle(), e);
}
List<BibEntry> citedBy = citationFetcher.searchCitedBy(cited);
relationsRepository.insertCitations(cited, citedBy);
}
return relationsRepository.readCitations(cited);
}
Expand Down
30 changes: 27 additions & 3 deletions ...abref/logic/importer/fetcher/citation/semanticscholar/SemanticScholarCitationFetcher.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,23 @@
import org.jabref.logic.importer.ImporterPreferences;
import org.jabref.logic.importer.fetcher.CustomizableKeyFetcher;
import org.jabref.logic.importer.fetcher.citation.CitationFetcher;
import org.jabref.logic.l10n.Localization;
import org.jabref.logic.net.URLDownload;
import org.jabref.logic.util.URLUtil;
import org.jabref.model.entry.BibEntry;

import com.google.gson.Gson;
import org.hisp.dhis.jsontree.JsonMixed;
import org.hisp.dhis.jsontree.JsonNode;
import org.jspecify.annotations.NonNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class SemanticScholarCitationFetcher implements CitationFetcher, CustomizableKeyFetcher {
public static final String FETCHER_NAME = "Semantic Scholar Citations Fetcher";

private static final Logger LOGGER = LoggerFactory.getLogger(SemanticScholarCitationFetcher.class);

private static final String SEMANTIC_SCHOLAR_API = "https://api.semanticscholar.org/graph/v1/";

private static final Gson GSON = new Gson();
Expand All @@ -27,8 +35,8 @@ public SemanticScholarCitationFetcher(ImporterPreferences importerPreferences) {
this.importerPreferences = importerPreferences;
}

public String getAPIUrl(String entry_point, BibEntry entry) {
return SEMANTIC_SCHOLAR_API + "paper/" + "DOI:" + entry.getDOI().orElseThrow().asString() + "/" + entry_point
public String getAPIUrl(String entryPoint, BibEntry entry) {
return SEMANTIC_SCHOLAR_API + "paper/" + "DOI:" + entry.getDOI().orElseThrow().asString() + "/" + entryPoint
+ "?fields=" + "title,authors,year,citationCount,referenceCount,externalIds,publicationTypes,abstract,url"
+ "&limit=1000";
}
Expand All @@ -42,6 +50,7 @@ public List<BibEntry> searchCitedBy(BibEntry entry) throws FetcherException {
URL citationsUrl;
try {
citationsUrl = URLUtil.create(getAPIUrl("citations", entry));
LOGGER.debug("Cited URL {} ", citationsUrl);
} catch (MalformedURLException e) {
throw new FetcherException("Malformed URL", e);
}
Expand All @@ -66,15 +75,30 @@ public List<BibEntry> searchCitedBy(BibEntry entry) throws FetcherException {
URL referencesUrl;
try {
referencesUrl = URLUtil.create(getAPIUrl("references", entry));
LOGGER.debug("Citing URL {} ", referencesUrl);
} catch (MalformedURLException e) {
throw new FetcherException("Malformed URL", e);
}

URLDownload urlDownload = new URLDownload(referencesUrl);
importerPreferences.getApiKey(getName()).ifPresent(apiKey -> urlDownload.addHeader("x-api-key", apiKey));
ReferencesResponse referencesResponse = GSON.fromJson(urlDownload.asString(), ReferencesResponse.class);
String response = urlDownload.asString();
ReferencesResponse referencesResponse = GSON.fromJson(response, ReferencesResponse.class);

if (referencesResponse.getData() == null) {
JsonNode json = JsonNode.of(response);
JsonNode disclaimerJson = json.getOrNull("citingPaperInfo.openAccessPdf.disclaimer");
Copy link
Member

@Siedlerchr Siedlerchr Jul 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use kong's unirest json that allows you to use optionals

            JSONObject response = JsonReader.toJsonObject(inputStream);


      JSONObject journal = journalInfo.getJSONObject("journal");
                    Optional.ofNullable(journal.optString("title")).ifPresent(title -> entry.setField(StandardField.JOURNAL, title));
                    ```

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chaining will be very ugly here?! Sure that we want to have 4 Optional.ofNullable calls? Or some chaining of null checks?

if (disclaimerJson != null) {
JsonMixed disclaimerNode = JsonMixed.of(disclaimerJson);
if (disclaimerNode.isString()) {
String disclaimer = disclaimerNode.string();
LOGGER.debug("Received a disclaimer from Semantic Scholar: {}", disclaimer);
if (disclaimer.contains("'references'")) {
throw new FetcherException(Localization.lang("Restricted access to references: %0", disclaimer));
}
}
}

return List.of();
}

Expand Down
10 changes: 7 additions & 3 deletions jablib/src/main/resources/l10n/JabRef_en.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1697,6 +1697,12 @@ Open\ one\ before\ citing.=Open one before citing.
Select\ one\ before\ citing.=Select one before citing.
Select\ some\ before\ citing.=Select some before citing.

Citation\ relations=Citation relations
Show\ articles\ related\ by\ citation=Show articles related by citation
Error\ while\ fetching\ cited\ entries\:\ %0=Error while fetching cited entries: %0
Error\ while\ fetching\ citing\ entries\:\ %0=Error while fetching citing entries: %0
Restricted\ access\ to\ references\:\ %0=Restricted access to references: %0

Found\ identical\ ranges=Found identical ranges
Found\ overlapping\ ranges=Found overlapping ranges
Found\ touching\ ranges=Found touching ranges
Expand Down Expand Up @@ -2809,9 +2815,7 @@ Restart\ search=Restart search
Cancel\ search=Cancel search
Select\ entry=Select entry
Search\ aborted!=Search aborted!
Citation\ relations=Citation relations
Show\ articles\ related\ by\ citation=Show articles related by citation
Error\ while\ fetching\ citing\ entries\:\ %0=Error while fetching citing entries: %0

Help\ on\ external\ applications=Help on external applications
Identifier-based\ Web\ Search=Identifier-based Web Search

Expand Down
Loading
Loading