Added some extra handling for outcomes (#864)

* Added some extra handling for outcomes
* Prevent clicked notifications in the foreground from being apart of the current session
* Added override functionality before session 30 second min, only lower sessions can be overridden by upper sessions
* Min time for focus and session is only in OneSignal.java
* Caching current session now so when the user quits the app and comes back we might want to stay in the current session
* Made the 5 second direct session timer back to 30 seconds

* Unit test was failing after adjusting outcomes work
* Needed to pause activity to background the app
* No DIRECT session could come from foreground click

* Updating unit tests for outcomes
* Changed attribution window code to use System.currentTimeMillis()
* Added test scenarios for new sessions, override sessions, and attribution window
* Changed some formatting in current outcome unit tests

* Added AppEntryState tracking and changing outcome flags for unit tests
* AppEntryAction is an enum with 3 states: APP_OPEN, APP_CLOSE, NOTIFICATION_OPEN
* appEntryState keeps track form OneSignal.java how we entered an application and will decide what type of session we currently are involved in
* Modified OutcomeRemoteParams in unit tests so we can choose our own flags for testing purposes

* Added default for appEntryState and cleaned up session checking

* Fixed comment for session restart

* Fixed checks in OSSessionManager
* We want to check session and data in separate places sometimes
* Now we have hasDirectNotification and hasIndirectNotifications along side isDirectSession and isIndirectSession

* Added helpers for checking appEntryState and session
* Methods for returning correct boolean associated with state check for AppEntryAction enum and Session enum
* Cleaned up tests and code so they use enum helpers

* Added OutcomeEventIntegrationTests
* Made a package private helper for getting OSSessionManager from OneSignal used to help test integration of outcomes in app sessions and focus
* Cleaned up outcomes integration tests in MainOneSignalClassRunner

* Move on_focus logic into it's own class

* Moved parts of on_focus logic from OneSignal and OneSignalSyncServiceUtils to a new FocusTimeController class
  - Good amount as also refactored in the move!
  - No functionally changes yet
* FocusTimeController will be split up in future commits, there is a large TODO block at the top of what is left.
* The shouldSyncPendingChangesFromSyncService test fails but is the only one.
  - Since a number of other changes are happening to FocusTimeController in this branch I held off on fixing this one.

* Tracking attributed time on session ending

* Refactored OSSessionManager.setSession to require notificationId(s).
  - This way it can fire off an event of the before state so the on_focus can subscribe and know the session just ended to make the call.
  - This means notificationId and notificationIds will only be set via the setSession
* WIP notes;
*   * Still need to add a number of tests.

* Fixed cold start on_focus sync

* Moved the initialization of OSSessionManager to set setAppContext as it need this to load any caches values

* Misc clean up based on PR review.

* Added case where indirect goes to indirect on new session
* Added unit testing for this scenario
* Also cleaned up OutcomeEventIntegrationTests so its easier to click and receive notifications
* Minor clean up with comments and naming

* Update OutcomesUtils.java

* Forgot to refactor name for method where its being used

* Removed any public where not needed in OneSignal.java
* Also added null check for sessionManager when clicking notif

* on_focus included background time on click

* Fixed case where background time could be included when tapping on a notification in the background.
   - To prevent this we clear the time stamp when the app goes into the background.

* Added package private helper for getting OutcomeSettings
* Fixed unit tests using OutcomeSettings
* OutcomeSettings needed to be package private

* Added test to validate on_focus is sent when overriding session
* Example is DIRECT session overriding an INDIRECT session
* Being in an INDIRECT session, leaving the app and clicking a notification should trigger a DIRECT session and the on_focus job for the INDIRECT session should fire

* Fixed email creating double sessions for outcomes

* Fixed session upgrade from indirect to direct

* Fixed testDirectSession_willOverrideIndirectSession_whenAppIsInBackground
* Some clean up to FocusTimeController + attemptSessionUpgrade comments

* Fixed OSSessionManager to be package-private

* Cleaned up completed TODOs

* Misc clean up

* Added getters for initDone and appEntryState

Author: mikechoch

OneSignalSDK/onesignal/src/main/java/com/onesignal/FocusTimeController.java

@@ -0,0 +1,267 @@
+package com.onesignal;
+
+import android.os.SystemClock;
+import android.support.annotation.NonNull;
+import android.support.annotation.Nullable;
+import android.support.annotation.WorkerThread;
+
+import org.json.JSONException;
+import org.json.JSONObject;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+
+/**
+ * Entry points that could cause on_focus to fire:
+ * 1. OSSessionManager.session changed (onSessionEnded) - Send any attributed session time
+ * 2. App is foregrounded (appForegrounded) - Set start focused time
+ * 3. App is backgrounded (appBackgrounded) - Kick off job to sync when session ends
+ */
+
+
+class FocusTimeController {
+   // Only present if app is currently in focus.
+   @Nullable private Long timeFocusedAtMs;
+
+   private static FocusTimeController sInstance;
+
+   private List<FocusTimeProcessorBase> focusTimeProcessors =
+      Arrays.asList(new FocusTimeProcessorUnattributed(), new FocusTimeProcessorAttributed());
+
+   private enum FocusEventType {
+      BACKGROUND,
+      END_SESSION
+   }
+
+   private FocusTimeController() { }
+   public static synchronized FocusTimeController getInstance() {
+      if (sInstance == null)
+         sInstance = new FocusTimeController();
+      return sInstance;
+   }
+
+   void appForegrounded() {
+      timeFocusedAtMs = SystemClock.elapsedRealtime();
+   }
+
+   void appBackgrounded() {
+      giveProcessorsValidFocusTime(OneSignal.getSessionManager().getSessionResult(), FocusEventType.BACKGROUND);
+      timeFocusedAtMs = null;
+   }
+
+   void onSessionEnded(@NonNull OSSessionManager.SessionResult lastSessionResult) {
+      final FocusEventType focusEventType = FocusEventType.END_SESSION;
+      boolean hadValidTime = giveProcessorsValidFocusTime(lastSessionResult, focusEventType);
+
+      // If there is no in focus time to be added we just need to send the time from the last session that just ended.
+      if (!hadValidTime) {
+         for (FocusTimeProcessorBase focusTimeProcessor : focusTimeProcessors)
+            focusTimeProcessor.sendUnsentTimeNow(focusEventType);
+      }
+   }
+
+   void doBlockingBackgroundSyncOfUnsentTime() {
+      if (OneSignal.isForeground())
+         return;
+
+      for(FocusTimeProcessorBase focusTimeProcessor : focusTimeProcessors)
+         focusTimeProcessor.syncUnsentTimeFromSyncJob();
+   }
+
+   private boolean giveProcessorsValidFocusTime(@NonNull OSSessionManager.SessionResult lastSessionResult, @NonNull FocusEventType focusType) {
+      Long timeElapsed = getTimeFocusedElapsed();
+      if (timeElapsed == null)
+        return false;
+
+      for(FocusTimeProcessorBase focusTimeProcessor : focusTimeProcessors)
+         focusTimeProcessor.addTime(timeElapsed, lastSessionResult.session, focusType);
+      return true;
+   }
+
+   // Get time past since app was put into focus.
+   // Will be null if time is invalid or 0
+   private @Nullable Long getTimeFocusedElapsed() {
+      // timeFocusedAtMs is cleared when the app goes into the background so we don't have a focus time
+      if (timeFocusedAtMs == null)
+         return null;
+
+      long timeElapsed = (long)(((SystemClock.elapsedRealtime() - timeFocusedAtMs) / 1_000d) + 0.5d);
+
+      // Time is invalid if below 1 or over a day
+      if (timeElapsed < 1 || timeElapsed > 86_400)
+         return null;
+      return timeElapsed;
+   }
+
+   private static class FocusTimeProcessorUnattributed extends FocusTimeProcessorBase {
+      FocusTimeProcessorUnattributed() {
+         MIN_ON_FOCUS_TIME_SEC = 60;
+         PREF_KEY_FOR_UNSENT_TIME = OneSignalPrefs.PREFS_GT_UNSENT_ACTIVE_TIME;
+      }
+
+      protected boolean timeTypeApplies(@NonNull OSSessionManager.Session session) {
+         return session.isUnattributed() || session.isDisabled();
+      }
+
+      protected void sendTime(@NonNull FocusEventType focusType) {
+         // We only need to send unattributed focus time when the app goes out of focus.
+         if (focusType.equals(FocusEventType.END_SESSION))
+            return;
+
+         syncUnsentTimeOnBackgroundEvent();
+      }
+   }
+
+   private static class FocusTimeProcessorAttributed extends FocusTimeProcessorBase {
+      FocusTimeProcessorAttributed() {
+         MIN_ON_FOCUS_TIME_SEC = 1;
+         PREF_KEY_FOR_UNSENT_TIME = OneSignalPrefs.PREFS_OS_UNSENT_ATTRIBUTED_ACTIVE_TIME;
+      }
+
+      protected boolean timeTypeApplies(@NonNull OSSessionManager.Session session) {
+         return session.isAttributed();
+      }
+
+      protected void additionalFieldsToAddToOnFocusPayload(@NonNull JSONObject jsonBody) {
+         OneSignal.getSessionManager().addSessionNotificationsIds(jsonBody);
+      }
+
+      protected void sendTime(@NonNull FocusEventType focusType) {
+         if (focusType.equals(FocusEventType.END_SESSION))
+            syncOnFocusTime();
+         else
+            OneSignalSyncServiceUtils.scheduleSyncTask(OneSignal.appContext);
+      }
+   }
+
+   private static abstract class FocusTimeProcessorBase {
+      // These values are set by child classes that inherit this base class
+      protected long MIN_ON_FOCUS_TIME_SEC;
+      protected @NonNull String PREF_KEY_FOR_UNSENT_TIME;
+
+      protected abstract boolean timeTypeApplies(@NonNull OSSessionManager.Session session);
+      protected abstract void sendTime(@NonNull FocusEventType focusType);
+
+      @Nullable private Long unsentActiveTime = null;
+
+      private long getUnsentActiveTime() {
+         if (unsentActiveTime == null) {
+            unsentActiveTime = OneSignalPrefs.getLong(
+               OneSignalPrefs.PREFS_ONESIGNAL,
+               PREF_KEY_FOR_UNSENT_TIME,
+               0
+            );
+         }
+         OneSignal.Log(OneSignal.LOG_LEVEL.DEBUG, this.getClass().getSimpleName() + ":getUnsentActiveTime: " + unsentActiveTime);
+         return unsentActiveTime;
+      }
+
+      private void saveUnsentActiveTime(long time) {
+         unsentActiveTime = time;
+         OneSignal.Log(OneSignal.LOG_LEVEL.DEBUG, this.getClass().getSimpleName() + ":saveUnsentActiveTime: " + unsentActiveTime);
+         OneSignalPrefs.saveLong(
+            OneSignalPrefs.PREFS_ONESIGNAL,
+            PREF_KEY_FOR_UNSENT_TIME,
+            time
+         );
+      }
+
+      private void addTime(long time, @NonNull OSSessionManager.Session session, @NonNull FocusEventType focusType) {
+         if (!timeTypeApplies(session))
+            return;
+
+         long totalTime = getUnsentActiveTime() + time;
+         saveUnsentActiveTime(totalTime);
+         sendUnsentTimeNow(focusType);
+      }
+
+      private void sendUnsentTimeNow(FocusEventType focusType) {
+         if (!OneSignal.hasUserId())
+            return;
+
+         sendTime(focusType);
+      }
+
+      private boolean hasMinSyncTime() {
+         return getUnsentActiveTime() >= MIN_ON_FOCUS_TIME_SEC;
+      }
+
+      protected void syncUnsentTimeOnBackgroundEvent() {
+         if (!hasMinSyncTime())
+            return;
+         // Schedule this sync in case app is killed before completing
+         OneSignalSyncServiceUtils.scheduleSyncTask(OneSignal.appContext);
+         syncOnFocusTime();
+      }
+
+      private void syncUnsentTimeFromSyncJob() {
+         if (hasMinSyncTime())
+            syncOnFocusTime();
+      }
+
+      @NonNull private final AtomicBoolean runningOnFocusTime = new AtomicBoolean();
+      @WorkerThread
+      protected void syncOnFocusTime() {
+         if (runningOnFocusTime.get())
+            return;
+
+         synchronized (runningOnFocusTime) {
+            runningOnFocusTime.set(true);
+            if (hasMinSyncTime())
+               sendOnFocus(getUnsentActiveTime());
+            runningOnFocusTime.set(false);
+         }
+      }
+
+      private void sendOnFocusToPlayer(@NonNull String userId, @NonNull JSONObject jsonBody) {
+         OneSignalRestClient.ResponseHandler responseHandler = new OneSignalRestClient.ResponseHandler() {
+            @Override
+            void onFailure(int statusCode, String response, Throwable throwable) {
+               OneSignal.logHttpError("sending on_focus Failed", statusCode, throwable, response);
+            }
+
+            @Override
+            void onSuccess(String response) {
+               // TODO: PRE-EXISTING: This time is shared between the email + push player and
+               //          is cleared no matter which one is successful.
+               // TODO: PRE-EXISTING: This could be clearing time more then was persisted while the network call was in flight
+               saveUnsentActiveTime(0);
+            }
+         };
+         String url = "players/" + userId + "/on_focus";
+         OneSignalRestClient.postSync(url, jsonBody, responseHandler);
+      }
+
+      // Override Optional
+      protected void additionalFieldsToAddToOnFocusPayload(@NonNull JSONObject jsonBody) { }
+
+      private @NonNull JSONObject generateOnFocusPayload(long totalTimeActive) throws JSONException {
+         JSONObject jsonBody = new JSONObject()
+            .put("app_id", OneSignal.appId)
+            .put("type", 1) // Always 1, where this type means do NOT increase session_count
+            .put("state", "ping") // Always ping, other types are not used
+            .put("active_time", totalTimeActive)
+            .put("device_type", new OSUtils().getDeviceType());
+         OneSignal.addNetType(jsonBody);
+         return jsonBody;
+      }
+
+      private void sendOnFocus(long totalTimeActive) {
+         try {
+            JSONObject jsonBody = generateOnFocusPayload(totalTimeActive);
+            additionalFieldsToAddToOnFocusPayload(jsonBody);
+            sendOnFocusToPlayer(OneSignal.getUserId(), jsonBody);
+
+            // For email we omit additionalFieldsToAddToOnFocusPayload as we don't want to add
+            //   outcome fields which would double report the session time
+            if (OneSignal.hasEmailId())
+               sendOnFocusToPlayer(OneSignal.getEmailId(), generateOnFocusPayload(totalTimeActive));
+         }
+         catch (JSONException t) {
+            OneSignal.Log(OneSignal.LOG_LEVEL.ERROR, "Generating on_focus:JSON Failed.", t);
+         }
+      }
+   }
+}

OneSignalSDK/onesignal/src/main/java/com/onesignal/GenerateNotification.java

@@ -263,7 +263,7 @@ private static OneSignalNotificationBuilder getBaseOneSignalNotificationBuilder(
 
       if (notifJob.shownTimeStamp != null) {
          try {
-            notifBuilder.setWhen(notifJob.shownTimeStamp * 1000L);
+            notifBuilder.setWhen(notifJob.shownTimeStamp * 1_000L);
          } catch (Throwable t) {} // Can throw if an old android support lib is used.
       }
 

OneSignalSDK/onesignal/src/main/java/com/onesignal/JSONUtils.java

@@ -16,9 +16,12 @@
 
 
 class JSONUtils {
-    // Returns a JSONObject of the differences between cur and changedTo.
-    // If baseOutput is added changes will be applied to this JSONObject.
-    // includeFields will always be added to the returned JSONObject if they are in cur.
+
+    /**
+     * Returns a JSONObject of the differences between cur and changedTo.
+     * If baseOutput is added changes will be applied to this JSONObject.
+     * includeFields will always be added to the returned JSONObject if they are in cur.
+     */
     static JSONObject generateJsonDiff(JSONObject cur, JSONObject changedTo, JSONObject baseOutput, Set<String> includeFields) {
         if (cur == null)
             return null;
@@ -158,7 +161,9 @@ static JSONObject getJSONObjectWithoutBlankValues(JSONObject jsonObject, String
         return jsonObjectToMapNonNull(json);
     }
 
-    // Converts a JSONObject into a Map, same as above however does NOT accept null values
+    /**
+     * Converts a JSONObject into a Map, same as above however does NOT accept null values
+     */
     private static @NonNull Map<String, Object> jsonObjectToMapNonNull(@NonNull JSONObject object) throws JSONException {
         Map<String, Object> map = new HashMap<>();
         Iterator<String> keysItr = object.keys();
@@ -170,7 +175,9 @@ static JSONObject getJSONObjectWithoutBlankValues(JSONObject jsonObject, String
         return map;
     }
 
-    // Converts a JSONArray into a List, returns null if a null value is passed in.
+    /**
+     * Converts a JSONArray into a List, returns null if a null value is passed in.
+     */
     static @Nullable List<Object> jsonArrayToList(@Nullable JSONArray array) throws JSONException {
         if (array == null)
             return null;
@@ -187,8 +194,10 @@ static JSONObject getJSONObjectWithoutBlankValues(JSONObject jsonObject, String
         return list;
     }
 
-    // Digs into any nested JSONObject or JSONArray to convert them to a List or Map
-    // If object is another type is is returned back.
+    /**
+     * Digs into any nested JSONObject or JSONArray to convert them to a List or Map
+     * If object is another type is is returned back.
+     */
     private static @NonNull Object convertNestedJSONType(@NonNull Object value) throws JSONException {
         if (value instanceof JSONObject)
             return jsonObjectToMapNonNull((JSONObject)value);
@@ -197,4 +206,59 @@ static JSONObject getJSONObjectWithoutBlankValues(JSONObject jsonObject, String
         return value;
     }
 
+    /**
+     * Compare two JSONArrays too determine if they are equal or not
+     */
+    static boolean compareJSONArrays(JSONArray jsonArray1, JSONArray jsonArray2) {
+        // If both JSONArrays are null, they are equal
+        if (jsonArray1 == null && jsonArray2 == null)
+            return true;
+
+        // If one JSONArray is null but not the other, they are not equal
+        if (jsonArray1 == null || jsonArray2 == null)
+            return false;
+
+        // If one JSONArray is a different size then the other, they are not equal
+        if (jsonArray1.length() != jsonArray2.length())
+            return false;
+
+        try {
+            L1 : for (int i = 0; i < jsonArray1.length(); i++) {
+                for (int j = 0; j < jsonArray2.length(); j++) {
+                    Object obj1 = normalizeType(jsonArray1.get(i));
+                    Object obj2 = normalizeType(jsonArray2.get(j));
+                    // Make sure jsonArray1 current item exists somewhere inside jsonArray2
+                    // If item found continue looping
+                    if (obj1.equals(obj2))
+                        continue L1;
+                }
+
+                // Could not find current item from jsonArray1 inside jsonArray2, so they are not equal
+                return false;
+            }
+        } catch (JSONException e) {
+            e.printStackTrace();
+
+            // Exception thrown, return false
+            return false;
+        }
+
+        // JSONArrays are equal
+        return true;
+    }
+
+    // Converts Java types that are equivalent in the JSON format to the same types.
+    // This allows for assertEquals on two values from JSONObject.get to test values as long as it
+    //   returns in the same JSON output.
+    public static Object normalizeType(Object object) {
+        Class clazz = object.getClass();
+
+        if (clazz.equals(Integer.class))
+            return Long.valueOf((Integer)object);
+        if (clazz.equals(Float.class))
+            return Double.valueOf((Float)object);
+
+        return object;
+    }
+
 }

OneSignalSDK/onesignal/src/main/java/com/onesignal/NotificationBundleProcessor.java

@@ -274,7 +274,7 @@ static void markRestoredNotificationAsDismissed(NotificationGenerationJob notifi
    // Clean up old records after 1 week.
    static void deleteOldNotifications(SQLiteDatabase writableDb) {
       writableDb.delete(NotificationTable.TABLE_NAME,
-          NotificationTable.COLUMN_NAME_CREATED_TIME + " < " + ((System.currentTimeMillis() / 1000L) - 604800L),
+          NotificationTable.COLUMN_NAME_CREATED_TIME + " < " + ((System.currentTimeMillis() / 1_000L) - 604800L),
           null);
    }