Implement mutiple inputs and update mleap dependency version with 0.14.0

Author: My Name

Dockerfile

@@ -9,7 +9,7 @@ WORKDIR /sagemaker-sparkml-model-server
 
 RUN mvn clean package
 
-RUN cp ./target/sparkml-serving-2.3.jar /usr/local/lib/sparkml-serving-2.3.jar
+RUN cp ./target/sparkml-serving-2.4.jar /usr/local/lib/sparkml-serving-2.4.jar
 RUN cp ./serve.sh /usr/local/bin/serve.sh
 
 RUN chmod a+x /usr/local/bin/serve.sh

README.md

@@ -223,20 +223,20 @@ Calling `CreateModel` is required for creating a `Model` in SageMaker with this
 SageMaker works with Docker images stored in [Amazon ECR](https://aws.amazon.com/ecr/). SageMaker team has prepared and uploaded the Docker images for SageMaker SparkML Serving Container in all regions where SageMaker operates. 
 Region to ECR container URL mapping can be found below. For a mapping from Region to Region Name, please see [here](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html).
 
-* us-west-1 = 746614075791.dkr.ecr.us-west-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* us-west-2 = 246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.2
-* us-east-1 = 683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* us-east-2 = 257758044811.dkr.ecr.us-east-2.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ap-northeast-1 = 354813040037.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ap-northeast-2 = 366743142698.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ap-southeast-1 = 121021644041.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ap-southeast-2 = 783357654285.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ap-south-1 = 720646828776.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* eu-west-1 = 141502667606.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* eu-west-2 = 764974769150.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-sparkml-serving:2.2
-* eu-central-1 = 492215442770.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* ca-central-1 = 341280168497.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-sparkml-serving:2.2
-* us-gov-west-1 = 414596584902.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-sparkml-serving:2.2
+* us-west-1 = 746614075791.dkr.ecr.us-west-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* us-west-2 = 246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.4
+* us-east-1 = 683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* us-east-2 = 257758044811.dkr.ecr.us-east-2.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ap-northeast-1 = 354813040037.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ap-northeast-2 = 366743142698.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ap-southeast-1 = 121021644041.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ap-southeast-2 = 783357654285.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ap-south-1 = 720646828776.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* eu-west-1 = 141502667606.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* eu-west-2 = 764974769150.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-sparkml-serving:2.4
+* eu-central-1 = 492215442770.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* ca-central-1 = 341280168497.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-sparkml-serving:2.4
+* us-gov-west-1 = 414596584902.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-sparkml-serving:2.4
 
 With [SageMaker Python SDK](https://github.com/aws/sagemaker-python-sdk)
 ------------------------------------------------------------------------
@@ -263,7 +263,7 @@ First you need to ensure that have installed [Docker](https://www.docker.com/) o
 In order to build the Docker image, you need to run a single Docker command:
 
 ```
-docker build -t sagemaker-sparkml-serving:2.2 .
+docker build -t sagemaker-sparkml-serving:2.4 .
 ```
 
 #### Running the image locally
@@ -272,7 +272,7 @@ In order to run the Docker image, you need to run the following command. Please
 The command will start the server on port 8080 and will also pass the schema as an environment variable to the Docker container. Alternatively, you can edit the `Dockerfile` to add `ENV SAGEMAKER_SPARKML_SCHEMA=schema` as well before building the Docker image.
 
 ```
-docker run -p 8080:8080 -e SAGEMAKER_SPARKML_SCHEMA=schema -v /tmp/model:/opt/ml/model sagemaker-sparkml-serving:2.2 serve
+docker run -p 8080:8080 -e SAGEMAKER_SPARKML_SCHEMA=schema -v /tmp/model:/opt/ml/model sagemaker-sparkml-serving:2.4 serve
 ```
 
 #### Invoking with a payload
@@ -287,7 +287,7 @@ or
 curl -i -H "content-type:application/json" -d "{\"data\":[feature_1,\"feature_2\",feature_3]}" http://localhost:8080/invocations
 ```
 
-The `Dockerfile` can be found at the root directory of the package. SageMaker SparkML Serving Container tags the Docker images using the Spark major version it is compatible with. Right now, it only supports Spark 2.2 and as a result, the Docker image is tagged with 2.2. 
+The `Dockerfile` can be found at the root directory of the package. SageMaker SparkML Serving Container tags the Docker images using the Spark major version it is compatible with. Right now, it only supports Spark 2.4 and as a result, the Docker image is tagged with 2.4. 
 
 In order to save the effort of building the Docker image everytime you are making a code change, you can also install [Maven](http://maven.apache.org/) and run `mvn clean package` at your project root to verify if the code is compiling fine and unit tests are running without any issue. 
 
@@ -310,7 +310,7 @@ aws ecr get-login --region us-west-2 --registry-ids 246618743249 --no-include-em
 * Download the Docker image with the following command:
 
 ```
-docker pull 246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.2
+docker pull 246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.4
 ```
 
 For running the Docker image, please see the Running the image locally section from above.

ci/buildspec.yml

@@ -9,10 +9,10 @@ phases:
     commands:
     - echo Build started on `date`
     - echo Building the Docker image...
-    - docker build -t sagemaker-sparkml-serving:2.3 .
-    - docker tag sagemaker-sparkml-serving:2.3 515193369038.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.3
+    - docker build -t sagemaker-sparkml-serving:2.4 .
+    - docker tag sagemaker-sparkml-serving:2.4 515193369038.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.4
   post_build:
     commands:
     - echo Build completed on `date`
     - echo Pushing the Docker image...
-    - docker push 515193369038.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.3
+    - docker push 515193369038.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sparkml-serving:2.4

pom.xml

@@ -24,7 +24,7 @@
   <modelVersion>4.0.0</modelVersion>
   <groupId>org.amazonaws.sagemaker</groupId>
   <artifactId>sparkml-serving</artifactId>
-  <version>2.3</version>
+  <version>2.4</version>
   <build>
     <plugins>
       <plugin>
@@ -154,7 +154,7 @@
     <dependency>
       <groupId>ml.combust.mleap</groupId>
       <artifactId>mleap-runtime_2.11</artifactId>
-      <version>0.13.0</version>
+      <version>0.14.0</version>
     </dependency>
     <dependency>
       <groupId>org.apache.commons</groupId>

serve.sh

@@ -1,3 +1,3 @@
 #!/bin/bash
 # This is needed to make sure Java correctly detects CPU/Memory set by the container limits
-java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -jar /usr/local/lib/sparkml-serving-2.3.jar
+java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -jar /usr/local/lib/sparkml-serving-2.4.jar

src/main/java/com/amazonaws/sagemaker/controller/ServingController.java

@@ -21,6 +21,7 @@
 
 import com.amazonaws.sagemaker.dto.BatchExecutionParameter;
 import com.amazonaws.sagemaker.dto.DataSchema;
+import com.amazonaws.sagemaker.dto.SageMakerDataListObject;
 import com.amazonaws.sagemaker.dto.SageMakerRequestObject;
 import com.amazonaws.sagemaker.helper.DataConversionHelper;
 import com.amazonaws.sagemaker.helper.ResponseHelper;
@@ -43,6 +44,7 @@
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.http.HttpHeaders;
+import org.springframework.http.HttpStatus;
 import org.springframework.http.MediaType;
 import org.springframework.http.ResponseEntity;
 import org.springframework.web.bind.annotation.RequestBody;
@@ -148,6 +150,54 @@ public ResponseEntity<String> transformRequestCsv(@RequestBody final byte[] csvR
         }
     }
 
+    /**
+     * Implements the invocations POST API for application/jsonlines input
+     *
+     * @param jsonLines, lines of json values
+     * @param accept, accept parameter from request
+     * @return ResponseEntity with body as the expected payload JSON & proper statuscode based on the input
+     */
+    @RequestMapping(path = "/invocations", method = POST, consumes = AdditionalMediaType.APPLICATION_JSONLINES_VALUE)
+    public ResponseEntity<String> transformRequestJsonLines(@RequestBody final byte[] jsonLines,
+                                                            @RequestHeader(value = HttpHeaders.ACCEPT, required = false) final String accept) {
+        if (jsonLines == null) {
+            LOG.error("Input passed to the request is empty");
+            return ResponseEntity.noContent().build();
+        }
+        try {
+            final String acceptVal = this.retrieveAndVerifyAccept(accept);
+            final DataSchema schema = this.retrieveAndVerifySchema(null, mapper);
+            final String jsonStringLine = new String(jsonLines);
+
+            // Map list of inputs to DataList object
+            final SageMakerDataListObject sro = mapper.readValue(jsonStringLine, SageMakerDataListObject.class);
+            List<List<Object>> inputDatas = sro.getData();
+            List<ResponseEntity<String>> responseList = Lists.newArrayList();
+
+            // Process each input separately and add response to a list
+            final int inputDatasSize = inputDatas.size();
+            for (int idx = 0; idx < inputDatasSize; ++idx) {
+                ResponseEntity<String> response = this.processInputData(inputDatas.get(idx), schema, acceptVal);
+                responseList.add(response);
+            }
+
+            // Merge response body to a new output response
+            List<List<String>> bodyList = Lists.newArrayList();
+            HttpHeaders headers = null;
+            //combine body in responseList
+            for (ResponseEntity<String> response:responseList) {
+                HttpStatus statuscode = response.getStatusCode();
+                headers = response.getHeaders();
+                bodyList.add(Lists.newArrayList(response.getBody()));
+            }
+
+            return ResponseEntity.ok().headers(headers).body(bodyList.toString());
+        } catch (final Exception ex) {
+            LOG.error("Error in processing current request", ex);
+            return ResponseEntity.badRequest().body(ex.getMessage());
+        }
+    }
+
     @VisibleForTesting
     protected String retrieveAndVerifyAccept(final String acceptFromRequest) {
         final String acceptVal = checkEmptyAccept(acceptFromRequest) ? SystemUtils

src/main/java/com/amazonaws/sagemaker/dto/SageMakerDataListObject.java

@@ -0,0 +1,42 @@
+/*
+ *  Copyright 2010-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ *  Licensed under the Apache License, Version 2.0 (the "License").
+ *  You may not use this file except in compliance with the License.
+ *  A copy of the License is located at
+ *
+ *      http://aws.amazon.com/apache2.0
+ *
+ *  or in the "license" file accompanying this file. This file is distributed
+ *  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ *  express or implied. See the License for the specific language governing
+ *  permissions and limitations under the License.
+ *
+ */
+
+package com.amazonaws.sagemaker.dto;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.base.Preconditions;
+
+import java.util.List;
+
+/**
+ * Request object POJO to which input request in JSONLINES format will be mapped to by Spring (using Jackson). For sample
+ * input, please see test/resources/com/amazonaws/sagemaker/dto
+ */
+public class SageMakerDataListObject {
+
+    private List<List<Object>> data;
+
+    @JsonCreator
+    public SageMakerDataListObject(
+            @JsonProperty("data") final List<List<Object>> data) {
+        this.data = Preconditions.checkNotNull(data);
+    }
+
+    public List<List<Object>> getData() {
+        return data;
+    }
+}

src/test/java/com/amazonaws/sagemaker/controller/ServingControllerTest.java

@@ -230,6 +230,42 @@ public void testListValueMLeapThrowsExceptionCsvApi() {
         Assert.assertEquals(output.getBody(), "input data is not valid");
     }
 
+    @Test
+    public void testJsonLinesApiWithListInput() {
+        schemaInJson = "{\"input\":[{\"name\":\"test_name_1\",\"type\":\"int\"},{\"name\":\"test_name_2\","
+                + "\"type\":\"double\"},{\"name\":\"test_name_3\",\"type\":\"string\"}],\"output\":{\"name\":\"out_name\",\"type\":\"int\",\"struct\":\"vector\"}}";
+        List<Object> outputResponseForFirstInput = Lists.newArrayList(1, 2);
+        List<Object> outputResponseForSecondInput = Lists.newArrayList(3, 4);
+        final String expectOutput = "[[1,2], [3,4]]";
+
+        PowerMockito.when(SystemUtils.getEnvironmentVariable("SAGEMAKER_SPARKML_SCHEMA")).thenReturn(schemaInJson);
+        PowerMockito
+                .when(ScalaUtils.getJavaObjectIteratorFromArrayRow(Mockito.any(ArrayRow.class), Mockito.anyString()))
+                .thenReturn(outputResponseForFirstInput.iterator())
+                .thenReturn(outputResponseForSecondInput.iterator());
+        final ResponseEntity<String> output = controller.transformRequestJsonLines("{\"data\":[[1,2.0,\"TEST1\"], [2,3.0,\"TEST\"]]}".getBytes(), "text/csv");
+        Assert.assertEquals(expectOutput, output.getBody());
+    }
+
+    @Test
+    public void testJsonLinesApiWithListInputThorwsException() {
+        schemaInJson = "{\"input\":[{\"name\":\"test_name_1\",\"type\":\"int\"},{\"name\":\"test_name_2\","
+                + "\"type\":\"double\"},{\"name\":\"test_name_3\",\"type\":\"string\"}],\"output\":{\"name\":\"out_name\",\"type\":\"int\",\"struct\":\"vector\"}}";
+        PowerMockito.when(SystemUtils.getEnvironmentVariable("SAGEMAKER_SPARKML_SCHEMA")).thenReturn(schemaInJson);
+        PowerMockito
+                .when(ScalaUtils.getJavaObjectIteratorFromArrayRow(Mockito.any(ArrayRow.class), Mockito.anyString()))
+                .thenThrow(new RuntimeException("input data is not valid"));
+        final ResponseEntity<String> output = controller.transformRequestJsonLines("{\"data\":[[1,2.0,\"TEST1\"], [2,3.0,\"TEST\"]]}".getBytes(), "text/csv");
+        Assert.assertEquals(output.getStatusCode(), HttpStatus.BAD_REQUEST);
+        Assert.assertEquals(output.getBody(), "input data is not valid");
+    }
+
+    @Test
+    public void testJsonLinesApiWithNullInput() {
+        PowerMockito.when(SystemUtils.getEnvironmentVariable("SAGEMAKER_SPARKML_SCHEMA")).thenReturn(schemaInJson);
+        final ResponseEntity<String> output = controller.transformRequestJsonLines(null, "text/csv");
+        Assert.assertEquals(output.getStatusCode(), HttpStatus.NO_CONTENT);
+    }
 
     @Test
     public void testParseAcceptEmptyFromRequestEnvironmentPresent() {