Skip to content

Sophi4 627: upgrade kcl version to implement Asyn Kinesis Function #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

tgampiyush
Copy link

@tgampiyush tgampiyush commented Mar 11, 2021

This PR upgraded the KCL version in Snowplow enriched, plus updated the code from the master branch of the forked repo.

The reference of this PR is taken from Saeed's work - https://github.com/globeandmail/enrich/pull/1/files

Steps -

  1. Rebased the forked master branch from the upstream branch. ( latest tag/release version on upstream - upstream/release/1.4.2)

  2. Upgraded KinesisClient and AWSSdk version and lib for kinesisSdk,dynamoSdk,s3sdk and its related code in kinesis module to use AsynClien's.

Steps To Test -

  1. Created the PR to build a docker image for the new code.
  1. Deploy Enricher and Collector on Dev-QA to test the flow.
    https://rancher.sophi.cloud/p/c-hkh4v:p-wkjmt/workload/deployment:develop:snowplow-enricher

@tgampiyush tgampiyush requested a review from tgambhavya March 11, 2021 22:38
@tgampiyush tgampiyush self-assigned this Mar 11, 2021
@tgampiyush tgampiyush added enhancement New feature or request in review labels Mar 15, 2021
@tgampiyush tgampiyush changed the title Enhancement/sophi4 627 upgrade kcl Sophi4 627: upgrade kcl version to implement Asyn Kinesis Function Mar 15, 2021
build.sbt Outdated
@@ -45,7 +45,6 @@ lazy val common = project
Dependencies.Libraries.jaywayJsonpath,
Dependencies.Libraries.iabClient,
Dependencies.Libraries.yauaa,
Dependencies.Libraries.rhino,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something's wrong here - where did these dependencies come from that you had to delete? When I check the Snowplow Enrich master I don't see either of them https://github.com/snowplow/enrich/blob/master/build.sbt

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya not sure, I too checked the master branch and didn't find them, hence deleted them ( also they were causing the issue if present).
Need to do due-diligence to figure it out.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to fix this before you go any further. For example the build.sbt in your repo's master branch is from last June, while the version in the public Snowplow repo is from October, and has many changes from the June version.
Ideally we want to be able to submit these changes back to the Snowplow project as a PR for their repo, so we had better make sure we are basing the changes on their most recent release.

@tgampiyush tgampiyush reopened this Mar 18, 2021
@aahmedg aahmedg added invalid This doesn't seem right wontfix This will not be worked on labels Apr 19, 2021
Copy link

@aahmedg aahmedg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abandon this pr in favor of snowplow#288

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request in review invalid This doesn't seem right wontfix This will not be worked on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants