From 8591d0d05d75528e40669f668894cc564d022c63 Mon Sep 17 00:00:00 2001 From: Ian Scott Date: Mon, 24 Feb 2025 20:46:30 -0500 Subject: [PATCH 1/4] fix(testing): Fix broken package install during setup of github action for tests --- .github/workflows/tests.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 4eea80799..8569cb17a 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -32,6 +32,7 @@ jobs: - name: Install prerequisites run: | + sudo apt-get update sudo apt-get install -y pkg-config libxml2-dev libxmlsec1-dev libxmlsec1-openssl - name: set up docker From fda14aac662e3211c1ae27b18d74e70e3f124a1f Mon Sep 17 00:00:00 2001 From: Ian Scott Date: Tue, 25 Feb 2025 18:49:43 -0500 Subject: [PATCH 2/4] fix(notifications): Corrections to support links in welcome emails --- templates/security/email/welcome.html | 8 +++----- templates/security/email/welcome.txt | 4 ++-- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/templates/security/email/welcome.html b/templates/security/email/welcome.html index 2c715188a..cd111fd95 100644 --- a/templates/security/email/welcome.html +++ b/templates/security/email/welcome.html @@ -12,7 +12,7 @@

Welcome to Knowledge Commons Works!

{{ _('Confirm my account') }}

{% endif %} -

Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks, you can: +

Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks, you can:

-

Need additional assistance with KCWorks? Visit support.hcommons.org to view our full help guides.

+

KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback.

-

KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback.

- -

Need additional assistance with KCWorks? Visit support.hcommons.org to view our full help guides.

+

Need additional assistance with KCWorks? Visit https://support.hcommons.org to view our full help guides.

Happy sharing!
diff --git a/templates/security/email/welcome.txt b/templates/security/email/welcome.txt index 0e2cd7b08..89f411683 100644 --- a/templates/security/email/welcome.txt +++ b/templates/security/email/welcome.txt @@ -1,4 +1,4 @@ -Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks (https://kcworks.hcommons.org), you can: +Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks (https://works.hcommons.org), you can: - Mint DOIs for every work you upload - Share your works through feeds to Google Scholar, core.ac.uk, and other metadata aggregators @@ -6,7 +6,7 @@ Welcome to KCWorks, the next generation of the Knowledge Commons repository! Wit - Contribute to collections of similar works - Control the metadata you’d like to share about a work, and hide works that are outdated -KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback: https://msu.co1.qualtrics.com/jfe/form/SV_1G1u7r34E4IE8dw. +KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback: https://support.hcommons.org/contact-us/ Need additional assistance with KCWorks? Visit support.hcommons.org to view our full help guides. From 30e684420fdf2f695e091e71d0b70e19a58e30b5 Mon Sep 17 00:00:00 2001 From: Ian Scott Date: Tue, 25 Feb 2025 18:50:48 -0500 Subject: [PATCH 3/4] fix(testing): Disabling warning summaries in test runner --- site/run_tests.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/site/run_tests.sh b/site/run_tests.sh index 2ecfeb6a3..8ff65a044 100644 --- a/site/run_tests.sh +++ b/site/run_tests.sh @@ -54,9 +54,9 @@ eval "$(PIPENV_DOTENV_LOCATION=/Users/ianscott/Development/knowledge-commons-wor # Note: expansion of pytest_args looks like below to not cause an unbound # variable error when 1) "nounset" and 2) the array is empty. if [ ${#pytest_args[@]} -eq 0 ]; then - PIPENV_DOTENV_LOCATION=/Users/ianscott/Development/knowledge-commons-works/site/tests/.env pipenv run python -m pytest -vv + PIPENV_DOTENV_LOCATION=/Users/ianscott/Development/knowledge-commons-works/site/tests/.env pipenv run python -m pytest -vv --disable-warnings else - PIPENV_DOTENV_LOCATION=/Users/ianscott/Development/knowledge-commons-works/site/tests/.env pipenv run python -m pytest ${pytest_args[@]} + PIPENV_DOTENV_LOCATION=/Users/ianscott/Development/knowledge-commons-works/site/tests/.env pipenv run python -m pytest ${pytest_args[@]} --disable-warnings fi # python -m sphinx.cmd.build -qnN -b doctest docs docs/_build/doctest tests_exit_code=$? From 721dec3ddabbd50e1bfba6f17224c4202b097764 Mon Sep 17 00:00:00 2001 From: Ian Scott Date: Fri, 28 Feb 2025 20:16:18 -0500 Subject: [PATCH 4/4] feature(importer, notifications): Adding email notifications to new importer API; importer tests passing; raised allowed upload size limit in nginx configuration to allow large file uploads. --- .../nginx_containerized/conf.d/default.conf | 2 +- docker/nginx_production/conf.d/default.conf | 2 +- docs/source/README.md | 2 +- docs/source/api.md | 21 +- docs/source/changelog.md | 8 +- invenio.cfg | 9 +- site/kcworks/__init__.py | 2 +- .../invenio-record-importer-kcworks | 2 +- .../user-first-record.create.jinja | 2 +- site/pyproject.toml | 2 +- site/tests/api/test_accounts.py | 48 +++- site/tests/api/test_api_import.py | 210 +++++++++++++----- site/tests/api/test_api_notifications.py | 37 +-- site/tests/api/test_search_provisioning.py | 2 +- site/tests/conftest.py | 48 +--- site/tests/fixtures/communities.py | 26 ++- templates/security/email/welcome.html | 17 +- templates/security/email/welcome.txt | 6 +- templates/security/email/welcome_neh.html | 26 +++ templates/security/email/welcome_neh.txt | 16 ++ 20 files changed, 358 insertions(+), 130 deletions(-) create mode 100644 templates/security/email/welcome_neh.html create mode 100644 templates/security/email/welcome_neh.txt diff --git a/docker/nginx_containerized/conf.d/default.conf b/docker/nginx_containerized/conf.d/default.conf index 60eda1796..0eb4b6187 100644 --- a/docker/nginx_containerized/conf.d/default.conf +++ b/docker/nginx_containerized/conf.d/default.conf @@ -122,7 +122,7 @@ server { uwsgi_hide_header X-Session-ID; uwsgi_hide_header X-User-ID; # Max upload size for files is set to 50GB (configure as needed). - client_max_body_size 50G; + client_max_body_size 500G; } # Static content is served directly by nginx and not the application server. location /static { diff --git a/docker/nginx_production/conf.d/default.conf b/docker/nginx_production/conf.d/default.conf index 44dfdfbbc..bfa0f6fdc 100644 --- a/docker/nginx_production/conf.d/default.conf +++ b/docker/nginx_production/conf.d/default.conf @@ -100,7 +100,7 @@ server { uwsgi_request_buffering off; # Max upload size for files is set to 50GB (configure as needed). - client_max_body_size 50G; + client_max_body_size 500G; } # Static content is served directly by nginx and not the application server. location /static { diff --git a/docs/source/README.md b/docs/source/README.md index c712c0776..141612626 100644 --- a/docs/source/README.md +++ b/docs/source/README.md @@ -2,7 +2,7 @@ Knowledge Commons Works is a collaborative tool for storing and sharing academic research. It is part of Knowledge Commons and is built on an instance of the InvenioRDM repository system. -Version 0.3.5-beta8 +Version 0.3.7-beta10 ## Copyright diff --git a/docs/source/api.md b/docs/source/api.md index 125571965..9752569b5 100644 --- a/docs/source/api.md +++ b/docs/source/api.md @@ -149,12 +149,10 @@ This request must be made with a multipart/form-data request. The request body m | `review_required` | no | `text/plain` | A string representation of a boolean (either "true" or "false") indicating whether the work should be reviewed before publication. This setting is only relevant if the work is intended for publication in a collection that requires review. It will override the collection's usual review policy, since the work is being uploaded by a collection administrator. (Default: "true") | | `strict_validation` | no | `text/plain` | A string representation of a boolean (either "true" or "false") indicating whether the import request should be rejected if any validation errors are encountered. If this value is "false", the imported work will be created in KCWorks even if some of the provided metadata does not conform to the KCWorks metadata schema, provided these are not required fields. If this value is "true", the import request will be rejected if any validation errors are encountered. (Default: "true") | | `all_or_none` | no | `text/plain` | A string representation of a boolean (either "true" or "false") indicating whether the entire import request should be rejected if any of the works fail to be created (whether for validation errors, upload errors, or other reasons). If this value is "false", the import request will be accepted even if some of the works cannot be created. The response in this case will include a list of works that were successfully created and a list of errors for the works that failed to be created. (Default: "true") | +| `notify_record_owners` | no | `text/plain` | A string representation of a boolean (either "true" or "false") indicating whether the owners of the work should be notified by email of the work's creation. (Default: "true") | #### Identifying the owners of the work -The array of owners, if provided in a metadata object's `parent.access.owned_by` property, must include at least the full name and email address of the users to be added as owners of the work. If the user already has a Knowledge Commons account, their username should also be provided. Additional identifiers (e.g., ORCID) may be provided as well to help avoid duplicate accounts, since a KCWorks account will be created for each user if they do not already have one. -#### Identifying the owners of the work - The array of owners, if provided in a metadata object's `parent.access.owned_by` property, must include at least the full name and email address of the users to be added as owners of the work. If the user already has a Knowledge Commons account, their username should also be provided. Additional identifiers (e.g., ORCID) may be provided as well to help avoid duplicate accounts, since a KCWorks account will be created for each user if they do not already have one. | key | required | type | description | @@ -201,6 +199,23 @@ KCWorks will create an internal KCWorks account for each work owner who does not If an owner does not already belong to the collection to which the records are being imported, that owner will also be added to the collection's membership with the "reader" role. The allows them access to any records restricted to the collection's membership, but does not afford them any additional permissions. What it does mean is that collection managers will be able to see all of the work owners in the list of collection members on the collection's landing page. +#### Email notifications for work owners + +When a work is imported into a collection, the work owners will receive an email notification unless the `notify_record_owners` parameter is set to "false". This email will include a link to the work's landing page on KCWorks. The email subject line and the email template used for this notification are configurable on a collection-by-collection basis. Authorized organizations should discuss the desired content with the KCWorks team. + +For KCWorks developers: The configuration for this email is found in the config variable `RECORD_IMPORTER_COMMUNITIES` in the KCWorks instance's `invenio.cfg` file. This is a dictionary whose keys are the collection slugs and whose values are dictionaries with the following keys: + +- `email_subject_import`: The subject line for the email notification. +- `email_template_import`: The name of the Jinja2 template file to use for the email notification. These templates must be located in the `templates/security/email` directory. One template file with an `.html` extension and one with a `.txt` extension are required, with identical names apart from the extension. The name provided in the `email_template_import` key should be the filename without the `.html` or `.txt` extension. + +The template will receive the following variables: + +- `record`: A dictionary containing the metadata for the imported work. +- `community_page_url`: The URL of the collection's landing page on KCWorks. +- `kc_registration_link`: The URL of the Knowledge Commons registration page. +- `user`: The KCWorks User object for the user being notified. + + #### Identifying the work for import It is crucial that each work to be imported is assigned a unique identifier. This may be an identifier used internally by the importing organization, it may be a universally unique string such as a UUID, or it may be a universal identifier such as a DOI or a handle. In either case it must be unique across all works to be imported for the collection. This identifier will be used to identify the work in the response, and will be used to identify the work when checking for duplicate imports. diff --git a/docs/source/changelog.md b/docs/source/changelog.md index e0b3e9675..f61dfb3c3 100644 --- a/docs/source/changelog.md +++ b/docs/source/changelog.md @@ -4,7 +4,13 @@ # Changes -## 0.4.0-beta9 (2025-02-25) +## 0.3.7-beta10 (2025-03-01) + +- Importer + - Added email notifications for failed imports along with a new API flag to disable them. + + +## 0.3.6-beta9 (2025-02-25) - Importer - Added a new streamlined importer API. diff --git a/invenio.cfg b/invenio.cfg index 60b65c074..b1a55a132 100644 --- a/invenio.cfg +++ b/invenio.cfg @@ -261,6 +261,8 @@ COMMONS_API_REQUEST_PROTOCOL = os.getenv( KC_WORDPRESS_DOMAIN = os.getenv("INVENIO_KC_WORDPRESS_DOMAIN", "hcommons-dev.org") KC_HELP_URL = f"https://support.{KC_WORDPRESS_DOMAIN}" KC_WORKS_HELP_URL = f"{KC_HELP_URL}/kcworks/" +KC_HELP_EMAIL = "hello@hcommons.org" +KC_CONTACT_FORM_URL = f"{KC_HELP_URL}/contact-us/" KC_FAQ_URL = f"{KC_HELP_URL}/faqs/#kcworks-faq" KC_PROFILES_URL_BASE = f"https://{KC_WORDPRESS_DOMAIN}/members/" KC_REGISTER_URL = f"https://{KC_WORDPRESS_DOMAIN}/membership/" @@ -1642,7 +1644,12 @@ RECORD_IMPORTER_DATA_DIR = Path( RECORD_IMPORTER_LOGS_LOCATION = Path( os.getenv("INVENIO_LOGGING_FS_LOGFILE", "/opt/invenio/src/logs/invenio.log") ).parent - +RECORD_IMPORTER_COMMUNITIES = { + "neh": { + "email_subject_register": "Your NEH Open Access Deposit is Ready", + "email_template_register": "welcome_neh", + } +} # Invenio-Previewer # ================= diff --git a/site/kcworks/__init__.py b/site/kcworks/__init__.py index f445339e7..d05548629 100644 --- a/site/kcworks/__init__.py +++ b/site/kcworks/__init__.py @@ -18,4 +18,4 @@ """KCWorks customizations to InvenioRDM.""" -__version__ = "0.3.5-beta8" +__version__ = "0.3.7-beta10" diff --git a/site/kcworks/dependencies/invenio-record-importer-kcworks b/site/kcworks/dependencies/invenio-record-importer-kcworks index 7ca78a427..ecf187353 160000 --- a/site/kcworks/dependencies/invenio-record-importer-kcworks +++ b/site/kcworks/dependencies/invenio-record-importer-kcworks @@ -1 +1 @@ -Subproject commit 7ca78a4275dbac0801a1fcd86fbe21906b7a2fa8 +Subproject commit ecf1873538e51ae949182ddc2519f2738b0db9c8 diff --git a/site/kcworks/templates/semantic-ui/invenio_notifications/user-first-record.create.jinja b/site/kcworks/templates/semantic-ui/invenio_notifications/user-first-record.create.jinja index e611218b4..4a8c0f74b 100644 --- a/site/kcworks/templates/semantic-ui/invenio_notifications/user-first-record.create.jinja +++ b/site/kcworks/templates/semantic-ui/invenio_notifications/user-first-record.create.jinja @@ -36,7 +36,7 @@ {{ _("Draft ID") }} - {{ draft_id }} ({{ _("View draft") }}) + {{ draft_id }} ({{ _("View draft") }}) {# diff --git a/site/pyproject.toml b/site/pyproject.toml index 0b8ba3e2a..4b5b703f4 100644 --- a/site/pyproject.toml +++ b/site/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "kcworks" -version = "0.3.5-beta8" +version = "0.3.7-beta10" [project.optional-dependencies] tests = ["pytest-invenio>=2.1.0,<3.0.0"] diff --git a/site/tests/api/test_accounts.py b/site/tests/api/test_accounts.py index 685205e64..c3442f2da 100644 --- a/site/tests/api/test_accounts.py +++ b/site/tests/api/test_accounts.py @@ -1,10 +1,10 @@ -import copy from pprint import pformat import pytest import datetime from flask import Flask from invenio_accounts import current_accounts from invenio_accounts.models import User +from invenio_record_importer_kcworks.services.users import UsersHelper import json from kcworks.services.accounts.saml import ( knowledgeCommons_account_info, @@ -16,6 +16,7 @@ from requests_mock.adapter import _Matcher as Matcher from types import SimpleNamespace from typing import Callable, Optional + from ..fixtures.saml import idp_responses from ..fixtures.users import user_data_set, AugmentedUserFixture @@ -365,6 +366,16 @@ def test_account_register_on_login( assert mailbox[0].subject == "Welcome to KCWorks!" assert mailbox[0].recipients == [user_data["email"]] assert mailbox[0].sender == app.config["MAIL_DEFAULT_SENDER"] + assert "Welcome to Knowledge Commons Works!" in mailbox[0].html + assert ( + f"Welcome {user_data['email']} to Knowledge Commons Works," in mailbox[0].body + ) + assert user_data["email"] in mailbox[0].html + assert user_data["email"] in mailbox[0].body + assert app.config["KC_HELP_URL"] in mailbox[0].html + assert app.config["KC_HELP_URL"] in mailbox[0].body + assert app.config["KC_CONTACT_FORM_URL"] in mailbox[0].html + assert app.config["KC_CONTACT_FORM_URL"] in mailbox[0].body user: User = current_accounts.datastore.get_user_by_email(user_data["email"]) assert user.email == user_data["email"] @@ -394,3 +405,38 @@ def test_account_register_on_login( assert not any([r for r in user.roles if r.name not in expected_roles]) assert next_url == "https://localhost/next-url.com" + + +def test_create_user_via_importer( + running_app, + appctx, + db, + mailbox, + celery_worker, + search_clear: Callable, +) -> None: + """ + Test the creation of a user programmatically via the importer. + + Among other things, test that the correct welcome email is sent to the user. + """ + app: Flask = running_app.app + user = UsersHelper().create_invenio_user( + user_email="test@example.com", + full_name="Test User", + community_owner=[], + orcid="0000-0002-1825-0097", + other_user_ids=[ + {"identifier": "test", "scheme": "neh_user_id"}, + {"identifier": "test2", "scheme": "import_user_id"}, + ], + ) + assert user["user"] is not None + assert user["user"].email == "test@example.com" + assert user["user"].username is None + assert user["user"].user_profile.get("full_name") == "Test User" + assert user["user"].user_profile.get("identifier_kc_username") is None + assert user["user"].user_profile.get("identifier_orcid") == "0000-0002-1825-0097" + assert user["user"].user_profile.get("name_parts") is None + assert user["user"].user_profile.get("affiliations") is None + assert len(mailbox) == 0 diff --git a/site/tests/api/test_api_import.py b/site/tests/api/test_api_import.py index 5cec08561..50fb9c599 100644 --- a/site/tests/api/test_api_import.py +++ b/site/tests/api/test_api_import.py @@ -72,7 +72,7 @@ def check_result_primary_community(self, result: LoaderResult, community: dict): """Check the primary community of the result.""" assert result.primary_community["id"] == community["id"] assert result.primary_community["metadata"]["title"] == "My Community" - assert result.primary_community["slug"] == "my-community" + assert result.primary_community["slug"] == "neh" def check_result_existing_record(self, result: LoaderResult): """Check the existing record of the result.""" @@ -110,6 +110,7 @@ def check_result_assigned_owners( user_id: str, test_metadata: TestRecordMetadata, app, + mailbox, ): """Check the assigned owners of the result.""" owners = ( @@ -117,13 +118,11 @@ def check_result_assigned_owners( .get("access", {}) .get("owned_by") ) - app.logger.debug(f"check_result_assigned_owners: {pformat(owners)}") if owners and result.status == "new_record": owners = [ current_accounts.datastore.get_user_by_email(owner["email"]) for owner in owners ] - app.logger.debug(f"check_result_assigned_owners Users: {pformat(owners)}") assert result.assigned_owners == { "owner_email": owners[0].email, "owner_id": owners[0].id, @@ -140,6 +139,51 @@ def check_result_assigned_owners( for owner in owners[1:] ], } + assert len(mailbox) == len(owners) + for owner in owners: + owner_sent_mail = [m for m in mailbox if m.recipients == [owner.email]][ + 0 + ] + assert ( + owner_sent_mail.subject == "Your NEH Open Access Deposit is Ready" + ) + assert owner_sent_mail.recipients == [owner.email] + assert owner_sent_mail.sender == app.config["MAIL_DEFAULT_SENDER"] + assert "Your NEH Open Access Deposit is Ready" in owner_sent_mail.html + assert "Your NEH Open Access Deposit is Ready" in owner_sent_mail.body + assert ( + test_metadata.metadata_in["metadata"]["title"] + in owner_sent_mail.html + ) + assert ( + test_metadata.metadata_in["metadata"]["title"] + in owner_sent_mail.body + ) + assert ( + result.record_created["record_data"]["pids"]["doi"]["identifier"] + in owner_sent_mail.html + ) + assert ( + result.record_created["record_data"]["pids"]["doi"]["identifier"] + in owner_sent_mail.body + ) + assert ( + result.record_created["record_data"]["links"]["self_html"] + in owner_sent_mail.html + ) + assert ( + result.record_created["record_data"]["links"]["self_html"] + in owner_sent_mail.body + ) + assert ( + result.primary_community["links"]["self_html"] + in owner_sent_mail.html + ) + assert ( + result.primary_community["links"]["self_html"] + in owner_sent_mail.body + ) + elif result.status == "new_record": assert result.assigned_owners == { "owner_id": user_id, @@ -203,33 +247,32 @@ def test_import_records_loader_load( record_metadata, mock_send_remote_api_update_fixture, celery_worker, + mailbox, ): app = running_app.app # find the resource type id for "textDocument" - rt = current_vocabulary_service.read( - system_identity, - id_=("resourcetypes", "textDocument-journalArticle"), - ) - app.logger.debug(f"textDocument rec: {pformat(rt.to_dict())}") + # rt = current_vocabulary_service.read( + # system_identity, + # id_=("resourcetypes", "textDocument-journalArticle"), + # ) - Vocabulary.index.refresh() + # Vocabulary.index.refresh() # Search for all resourcetypes - search_result = current_vocabulary_service.search( - system_identity, - type="resourcetypes", - ) - app.logger.debug(f"search_result: {pformat(search_result.to_dict())}") + # search_result = current_vocabulary_service.search( + # system_identity, + # type="resourcetypes", + # ) # Get the hits from the search result - resource_types = search_result.to_dict()["hits"]["hits"] + # resource_types = search_result.to_dict()["hits"]["hits"] # Print each resource type - for rt in resource_types: - app.logger.debug( - f"resource type: ID: {rt['id']}, Title: {rt['title']['en']}" - ) + # for rt in resource_types: + # app.logger.debug( + # f"resource type: ID: {rt['id']}, Title: {rt['title']['en']}" + # ) # Get the email of the first owner of the record if owners are specified owners = ( @@ -245,7 +288,7 @@ def test_import_records_loader_load( identity.provides.add(authenticated_user) login_user(u.user) - community_record = minimal_community_factory(owner=user_id) + community_record = minimal_community_factory(owner=user_id, slug="neh") community = community_record.to_dict() test_metadata = record_metadata( @@ -290,7 +333,7 @@ def test_import_records_loader_load( community.update({"links": {}}) # FIXME: Why are links not expanded? self.check_result_community_review_result(result, community, test_metadata) - self.check_result_assigned_owners(result, user_id, test_metadata, app) + self.check_result_assigned_owners(result, user_id, test_metadata, app, mailbox) self.check_result_added_to_collections(result) self.check_result_errors(result) @@ -421,6 +464,7 @@ def test_import_records_loader_load( record_metadata_with_files, mock_send_remote_api_update_fixture, celery_worker, + mailbox, ): app = running_app.app @@ -429,7 +473,6 @@ def test_import_records_loader_load( system_identity, id_=("resourcetypes", "textDocument-journalArticle"), ) - app.logger.debug(f"textDocument rec: {pformat(rt.to_dict())}") Vocabulary.index.refresh() # Search for all resourcetypes @@ -437,7 +480,6 @@ def test_import_records_loader_load( system_identity, type="resourcetypes", ) - app.logger.debug(f"search_result: {pformat(search_result.to_dict())}") # Get the hits from the search result resource_types = search_result.to_dict()["hits"]["hits"] @@ -454,7 +496,7 @@ def test_import_records_loader_load( identity.provides.add(authenticated_user) login_user(u.user) - community_record = minimal_community_factory(owner=user_id) + community_record = minimal_community_factory(owner=user_id, slug="neh") community = community_record.to_dict() file_paths = [ @@ -565,7 +607,7 @@ def test_import_records_loader_load( self.check_result_record_created(result, test_metadata) self.check_result_uploaded_files(result) self.check_result_community_review_result(result, community, test_metadata) - self.check_result_assigned_owners(result, user_id, test_metadata, app) + self.check_result_assigned_owners(result, user_id, test_metadata, app, mailbox) self.check_result_added_to_collections(result) self.check_result_submitted(result, test_metadata, app) self.check_result_errors(result) @@ -846,16 +888,7 @@ def _check_owners_in_community( target_roles = ( ["reader"] if user.id != uploader_id else ["curator", "manager", "owner"] ) - self.app.logger.debug( - f"user.id: {user.id}, type(user.id): {type(user.id)}, uploader_id: {uploader_id}, type(uploader_id): {type(uploader_id)}, target_roles: {target_roles}" - ) - self.app.logger.debug( - f"community_members: {pformat([(m.user_id, type(m.user_id), m.role) for m in community_members])}" - ) matching_ids = [m for m in community_members if m.user_id == user.id] - self.app.logger.debug( - f"matching_ids: {pformat([(m.user_id, m.role) for m in matching_ids])}" - ) assert matching_ids assert matching_ids[0].role in target_roles assert len(matching_ids) == 1 @@ -866,16 +899,15 @@ def _check_owners( expected: TestRecordMetadataWithFiles, uploader_id: str, community_id: str, + community_slug: str, + mailbox, + mocker, ): expected_owners = ( expected.metadata_in.get("parent", {}).get("access", {}).get("owned_by") ) if expected_owners: community_members = Member.get_members(community_id) - self.app.logger.debug(f"community_members: {pformat(community_members)}") - self.app.logger.debug( - f"community_members: {pformat([(m.user_id, m.role) for m in community_members])}" - ) first_expected_owner = expected.metadata_in["parent"]["access"]["owned_by"][ 0 ] @@ -884,7 +916,7 @@ def _check_owners( ) assert first_actual_owner.email == first_expected_owner["email"] self._check_owners_in_community( - community_members, first_actual_owner, uploader_id + community_members, first_actual_owner, int(uploader_id) ) if len(expected_owners) > 1: other_expected_owners = expected.metadata_in["parent"]["access"][ @@ -949,8 +981,67 @@ def _check_owners( # make sure they were added to the community # as reader (unless they are the uploader) self._check_owners_in_community( - community_members, user, uploader_id + community_members, user, int(uploader_id) ) + + self.app.logger.debug( + f"mailbox: {pformat([m.recipients for m in mailbox])}" + ) + self.app.logger.debug( + f"mailbox: {pformat([email.body for email in mailbox])}" + ) + # multiple records created with the one mailbox + if not self.by_api: # can't detect async email sending in test + mails_for_record = [ + m for m in mailbox if actual_metadata["metadata"]["title"] in m.html + ] + assert len(mails_for_record) == len(expected_owners) + for owner in expected_owners: + owner_sent_mail = [ + m for m in mails_for_record if m.recipients == [owner["email"]] + ][0] + self.app.logger.debug( + f"owner to send mail: {pformat(owner['email'])}" + ) + self.app.logger.debug( + f"owner_sent_mail recipients: " + f"{pformat(owner_sent_mail.recipients)}" + ) + assert ( + owner_sent_mail.subject + == "Your NEH Open Access Deposit is Ready" + ) + assert owner_sent_mail.recipients == [owner["email"]] + assert ( + owner_sent_mail.sender == self.app.config["MAIL_DEFAULT_SENDER"] + ) + assert ( + "Your NEH Open Access Deposit is Ready" in owner_sent_mail.html + ) + assert ( + "Your NEH Open Access Deposit is Ready" in owner_sent_mail.body + ) + assert actual_metadata["metadata"]["title"] in owner_sent_mail.html + assert actual_metadata["metadata"]["title"] in owner_sent_mail.body + assert ( + actual_metadata["pids"]["doi"]["identifier"] + in owner_sent_mail.html + ) + assert ( + actual_metadata["pids"]["doi"]["identifier"] + in owner_sent_mail.body + ) + assert actual_metadata["links"]["self_html"] in owner_sent_mail.html + assert actual_metadata["links"]["self_html"] in owner_sent_mail.body + assert ( + f"{self.app.config['SITE_UI_URL']}/collections/{community_slug}" + in owner_sent_mail.html + ) + assert ( + f"{self.app.config['SITE_UI_URL']}/collections/{community_slug}" + in owner_sent_mail.body + ) + else: assert actual_metadata["parent"]["access"]["owned_by"] == { "user": uploader_id @@ -964,6 +1055,8 @@ def _check_successful_import( expected: TestRecordMetadataWithFiles, community: dict, uploader_id: str, + mailbox, + mocker, ): assert self.app actual_metadata = actual.get("metadata") @@ -1002,7 +1095,15 @@ def _check_successful_import( f["checksum"] = actual_metadata["files"]["entries"][k]["checksum"] assert expected.compare_published(actual_metadata) - self._check_owners(actual_metadata, expected, uploader_id, community["id"]) + self._check_owners( + actual_metadata, + expected, + uploader_id, + community["id"], + community["slug"], + mailbox, + mocker, + ) # Check the record in the database record_id1 = actual_metadata.get("id") @@ -1022,6 +1123,8 @@ def check_result_data( metadata_sources: list, community: dict, uploader_id: str, + mailbox, + mocker, ) -> None: assert self.app expected_error_count = len([e for e in self.expected_errors if e]) @@ -1050,6 +1153,8 @@ def check_result_data( metadata_sources[idx], community, uploader_id, + mailbox, + mocker, ) def _do_api_import( @@ -1072,6 +1177,7 @@ def _do_api_import( "review_required": "true", "strict_validation": "true", "all_or_none": "true", + "notify_record_owners": "false", # Will error otherwise "files": file_streams, }, headers={ @@ -1090,6 +1196,8 @@ def test_import_records_service_load( user_factory, search_clear, mock_send_remote_api_update_fixture, + mailbox, + mocker, ): self.app = running_app.app u = user_factory(email="test@example.com", token=True, saml_id=None) @@ -1097,12 +1205,8 @@ def test_import_records_service_load( identity = get_identity(u.user) identity.provides.add(authenticated_user) - # FIXME: We need to actually create a KC account for the users - # assigned as owners, not just a KCWorks account. Or maybe send - # them an email with a link to create a KC account with the same - # email address? - community_record = minimal_community_factory( + slug="neh", owner=u.user.id, access=self.community_access_override, ) @@ -1115,7 +1219,7 @@ def test_import_records_service_load( submitter_identity, submitter_token = identity, u.allowed_token if not self.by_api: - login_user(submitter_identity.user) + login_user(submitter_identity.user) # type: ignore # Remember to close the file streams after the import is complete files, file_list, file_streams = self.files_to_upload @@ -1178,6 +1282,8 @@ def test_import_records_service_load( metadata_source_objects, community, user_id, + mailbox, + mocker, ) @@ -1267,7 +1373,7 @@ def expected_errors(self): ] -class TestImportAPIJournalArticle(BaseImportServiceTest): +class TestImportAPIJArticleSuccess(BaseImportServiceTest): """Test importing two journal articles via the API with no errors.""" @property @@ -1282,7 +1388,7 @@ def metadata_sources(self): ] -class BaseInsufficientPermissionsTest(TestImportAPIJournalArticle): +class BaseInsufficientPermissionsTest(TestImportAPIJArticleSuccess): """Base class for tests that check the API with insufficient permissions.""" def check_result_status(self, import_results: dict, status_code: Optional[int]): @@ -1348,15 +1454,13 @@ def make_submitter(self, user_factory, community_id): return new_user.user.id, new_user.allowed_token -class TestImportAPIJournalArticleErrorTitle(TestImportServiceJArticleErrorTitle): +class TestImportAPIJArticleErrorTitle(TestImportServiceJArticleErrorTitle): @property def by_api(self): return True -class TestImportAPIJournalArticleErrorMissingFile( - TestImportServiceJArticleErrorMissingFile -): +class TestImportAPIJArticleErrorMissingFile(TestImportServiceJArticleErrorMissingFile): @property def by_api(self): return True diff --git a/site/tests/api/test_api_notifications.py b/site/tests/api/test_api_notifications.py index 6fa747e53..aeddc6e1d 100644 --- a/site/tests/api/test_api_notifications.py +++ b/site/tests/api/test_api_notifications.py @@ -1481,19 +1481,23 @@ def test_notification_on_first_upload( f"Draft ID: {first_draft_id} ({app.config.get('SITE_UI_URL')}/records/" f"{first_draft_id})" in email.body ) + site_ui_url = app.config.get("SITE_UI_URL") + assert ( # fmt: off + f"Draft ID\n " + f'{first_draft_id} (View draft)\n ' in email.html + ) # fmt: on + assert f"Draft title: {metadata.draft['metadata']['title']}" in email.body assert ( - f"Draft ID: {first_draft_id} (" - f"View draft)" in email.html + f"Draft title\n {metadata.draft['metadata']['title']}" + in email.html ) - assert f"Draft title: {metadata.draft['metadata']['title']}" in email.body - assert f"Draft title: {metadata.draft['metadata']['title']}" in email.html assert f"User ID: {user_id}" in email.body - assert f"User ID: {user_id}" in email.html + assert f"User ID\n {user_id}" in email.html assert f"User email: {user_email}" in email.body - assert f"User email: {user_email}" in email.html + assert f"User email\n {user_email}" in email.html assert f"User name: {username}" in email.body - assert f"User name: {username}" in email.html + assert f"User name\n {username}" in email.html assert "A new user has created their first draft." in email.body assert "A new user has created their first draft." in email.html @@ -1526,15 +1530,22 @@ def test_notification_on_first_upload( ) assert email.sender == app.config["MAIL_DEFAULT_SENDER"] app.logger.debug(f"email.body: {pformat(email.body)}") + assert ( + f"Work ID\n " + f"{first_draft_id} (View work)\n ' in email.html + ) assert f"Work ID: {first_draft_id}" in email.body - assert f"Work ID: {first_draft_id}" in email.html + assert ( + f"Work title\n " + f"{metadata.draft['metadata']['title']}" in email.html + ) assert f"Work title: {metadata.draft['metadata']['title']}" in email.body - assert f"Work title: {metadata.draft['metadata']['title']}" in email.html assert f"User ID: {user_id}" in email.body - assert f"User ID: {user_id}" in email.html + assert f"User ID\n {user_id}" in email.html assert f"User email: {user_email}" in email.body - assert f"User email: {user_email}" in email.html + assert f"User email\n {user_email}" in email.html assert f"User name: {username}" in email.body - assert f"User name: {username}" in email.html + assert f"User name\n {username}" in email.html assert "A new user has published their first work." in email.body assert "A new user has published their first work." in email.html diff --git a/site/tests/api/test_search_provisioning.py b/site/tests/api/test_search_provisioning.py index 270c50d46..fead1ee1c 100644 --- a/site/tests/api/test_search_provisioning.py +++ b/site/tests/api/test_search_provisioning.py @@ -470,7 +470,7 @@ def test_trigger_community_provisioning( # Creation, # API operations should be prompted - actual_new = minimal_community_factory(admin.user.id) + actual_new = minimal_community_factory(admin.user.id, mock_search_api=False) assert actual_new["metadata"]["title"] == "My Community" assert requests_mock.call_count == 1 # user update at token login assert requests_mock.request_history[0].method == "POST" diff --git a/site/tests/conftest.py b/site/tests/conftest.py index 08f4b2818..33aed793f 100644 --- a/site/tests/conftest.py +++ b/site/tests/conftest.py @@ -1,14 +1,14 @@ from collections import namedtuple +from pprint import pformat import os from pathlib import Path import importlib import shutil import tempfile -from invenio_app.factory import create_app as create_ui_api +from invenio_app.factory import create_app as _create_app from invenio_queues import current_queues from invenio_search.proxies import current_search_client import jinja2 -from marshmallow import Schema, fields import pytest from .fixtures.identifiers import test_config_identifiers @@ -73,7 +73,6 @@ def _(x): "WTF_CSRF_ENABLED": False, "WTF_CSRF_METHODS": [], "RATELIMIT_ENABLED": False, - "APP_THEME": "semantic-ui", "APP_DEFAULT_SECURE_HEADERS": { "content_security_policy": {"default-src": []}, "force_https": False, @@ -136,28 +135,15 @@ def _(x): ) -class CustomUserProfileSchema(Schema): - """The default user profile schema.""" - - full_name = fields.String() - affiliations = fields.String() - name_parts = fields.String() - identifier_email = fields.String() - identifier_orcid = fields.String() - identifier_kc_username = fields.String() - unread_notifications = fields.String() - - -test_config["ACCOUNTS_USER_PROFILE_SCHEMA"] = CustomUserProfileSchema() - -# @pytest.fixture(scope="module") - # @pytest.fixture(scope="module") # def extra_entry_points() -> dict: # return { -# # 'invenio_db.models': [ -# # 'mock_module = mock_module.models', -# # ] +# "invenio_base.api_blueprints": [ +# "kcworks_templates = tests.fixtures.template_loader:template_blueprint_loader" +# ], +# "invenio_base.blueprints": [ +# "kcworks_templates = tests.fixtures.template_loader:template_blueprint_loader" +# ], # } @@ -346,20 +332,6 @@ def app( app_config, database, search, - # affiliations_v, - # awards_v, - # community_type_v, - # contributors_role_v, - # creators_role_v, - # date_type_v, - # description_type_v, - # funders_v, - # language_v, - # licenses_v, - # relation_type_v, - # resource_type_v, - # subject_v, - # title_type_v, template_loader, admin_roles, ): @@ -377,5 +349,5 @@ def app_config(app_config) -> dict: @pytest.fixture(scope="module") -def create_app(): - return create_ui_api +def create_app(instance_path, entry_points): + return _create_app diff --git a/site/tests/fixtures/communities.py b/site/tests/fixtures/communities.py index 70ac168cb..ce0f53638 100644 --- a/site/tests/fixtures/communities.py +++ b/site/tests/fixtures/communities.py @@ -155,7 +155,9 @@ def assemble_data() -> list[dict]: @pytest.fixture(scope="function") -def minimal_community_factory(app, db, user_factory, create_communities_custom_fields): +def minimal_community_factory( + app, db, user_factory, create_communities_custom_fields, requests_mock, monkeypatch +): """ Create a minimal community for testing. @@ -170,6 +172,7 @@ def create_minimal_community( access: dict = {}, custom_fields: dict = {}, members: dict = {"reader": [], "curator": [], "manager": [], "owner": []}, + mock_search_api: bool = True, ): """ Create a minimal community for testing. @@ -179,6 +182,27 @@ def create_minimal_community( If no owner is specified, a new user is created and used as the owner. """ + # Mock the search API for the community + if mock_search_api: + # Set up mock subscriber and intercept message to callback + monkeypatch.setenv("MOCK_SIGNAL_SUBSCRIBER", "True") + + search_api_url = list( + app.config["REMOTE_API_PROVISIONER_EVENTS"]["community"].keys() + )[0] + remote_response = { + "_internal_id": "1234AbCD?", # can't mock because set at runtime + "_id": "2E9SqY0Bdd2QL-HGeUuA34AbCD?", + "title": "My Community", + "primary_url": "http://works.kcommons.org/collections/my-community", + } + requests_mock.request( + "POST", + search_api_url, + json=remote_response, + headers={"Authorization": "Bearer 12345"}, + ) # noqa: E501 + if owner is None: owner = user_factory().user.id slug = slug or "my-community" diff --git a/templates/security/email/welcome.html b/templates/security/email/welcome.html index cd111fd95..9fccbe0e3 100644 --- a/templates/security/email/welcome.html +++ b/templates/security/email/welcome.html @@ -5,14 +5,9 @@

Welcome to Knowledge Commons Works!

-

{{ _('Welcome %(email)s!', email=user.email) }}

- {% if security.confirmable %} -

{{ _('You can confirm your email through the link below:') }}

-

{{ _('Confirm my account') }}

- {% endif %} -

Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks, you can: +

{{ _('Welcome %(email)s', email=user.email) }} to KCWorks, the open access repository for Knowledge Commons! With KCWorks, you can:

-

KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback.

+ {% if security.confirmable %} +

{{ _('You can confirm your email through the link below:') }}

+ +

{{ _('Confirm my account') }}

+ {% endif %} + +

KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback.

-

Need additional assistance with KCWorks? Visit https://support.hcommons.org to view our full help guides.

+

Need additional assistance with KCWorks? Visit {{ config.get('KC_HELP_URL') }} to view our full help guides.

Happy sharing!
diff --git a/templates/security/email/welcome.txt b/templates/security/email/welcome.txt index 89f411683..508c8352a 100644 --- a/templates/security/email/welcome.txt +++ b/templates/security/email/welcome.txt @@ -1,4 +1,4 @@ -Welcome to KCWorks, the next generation of the Knowledge Commons repository! With KCWorks (https://works.hcommons.org), you can: +{{ _('Welcome %(email)s', email=user.email) }} to Knowledge Commons Works, the open access repository for Knowledge Commons! With KCWorks ({{ config.get("SITE_UI_URL") }}), you can: - Mint DOIs for every work you upload - Share your works through feeds to Google Scholar, core.ac.uk, and other metadata aggregators @@ -6,9 +6,9 @@ Welcome to KCWorks, the next generation of the Knowledge Commons repository! Wit - Contribute to collections of similar works - Control the metadata you’d like to share about a work, and hide works that are outdated -KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback: https://support.hcommons.org/contact-us/ +KCWorks is currently in public beta, which means the repository is fully functional, but we’re still tweaking things behind the scenes. Use this form to provide us with feedback: {{ config.get("KC_CONTACT_FORM_URL") }} -Need additional assistance with KCWorks? Visit support.hcommons.org to view our full help guides. +Need additional assistance with KCWorks? Visit {{ config.get("KC_HELP_URL") }} to view our full help guides. Happy sharing! The Knowledge Commons Team diff --git a/templates/security/email/welcome_neh.html b/templates/security/email/welcome_neh.html new file mode 100644 index 000000000..317b827ba --- /dev/null +++ b/templates/security/email/welcome_neh.html @@ -0,0 +1,26 @@ + + + + + + +
+
+

Your NEH Open Access Deposit is Ready

+
+

Congratulations! Your NEH funded research has just been added to the NEH open access repository hosted on Knowledge Commons Works. Your work {{ record['metadata']['title'] }} is now available to the public and can be found at {{ record['links']['self_html'] }}.

+ +

Your research has been assigned a DOI, which is a permanent identifier for your work: {{ record['pids']['doi']['identifier'] }}. It is discoverable by search engines as well as from the NEH funded research search page and the global Knowledge Commons Works search page.

+ +

If you wish to manage your work yourself, and you have a Knowledge Commons account, you can log in. This will allow you to: +

    +
  • edit and update the metadata for your work
    • +
  • share your work with others
    • +
  • create updated versions of your work with new files
    • +
+

+ +

If you wish to manage your work yourself, and you do not have a Knowledge Commons account, you can create one at {{ config.get('KC_REGISTER_URL') }}. Be sure to use the same email address to which this email was sent when you create the Knowledge Commons account: {{ user["email"] }}.

+ +

Need additional assistance with Knowledge Commons Works? Visit {{ config.get('KC_HELP_URL') }} to view our full help guides or contact us at {{ config.get('KC_HELP_EMAIL') }} (or use the contact form)

+
\ No newline at end of file diff --git a/templates/security/email/welcome_neh.txt b/templates/security/email/welcome_neh.txt new file mode 100644 index 000000000..beec5bec8 --- /dev/null +++ b/templates/security/email/welcome_neh.txt @@ -0,0 +1,16 @@ + +Your NEH Open Access Deposit is Ready + +Congratulations! Your NEH funded research has just been added to the NEH open access repository ({{ community_page_url }}) hosted on Knowledge Commons Works. Your work "{{ record['metadata']['title'] }}" is now available to the public and can be found at {{ record['links']['self_html'] }}. + +Your research has been assigned a DOI, which is a permanent identifier for your work: {{ record['pids']['doi']['identifier'] }}. It is discoverable by search engines as well as from the NEH funded research search page and the global Knowledge Commons Works search page. + +If you wish to manage your work yourself, and you have a Knowledge Commons account, you can log in. This will allow you to: + +- edit and update the metadata for your work +- share your work with others +- create updated versions of your work with new files + +If you wish to manage your work yourself, and you do not have a Knowledge Commons account, you can create one at {{ config.get('KC_REGISTER_URL') }}. Be sure to use the same email address to which this email was sent when you create the Knowledge Commons account: {{ user['email'] }}. + +Need additional assistance with Knowledge Commons Works? Visit {{ config.get('KC_HELP_URL') }} to view our full help guides or contact us at {{ config.get('KC_HELP_EMAIL') }} (or use the contact form at {{ config.get('KC_CONTACT_FORM_URL') }}) \ No newline at end of file