|  | 
|  | 1 | +# Generic Importers | 
|  | 2 | + | 
|  | 3 | +## Implementing a Server | 
|  | 4 | + | 
|  | 5 | +To consume data from a generic importer, create a HTTP service in the language of your choice. | 
|  | 6 | +The service should accept POST requests on a given base URL, and kebab-case sub-path specific to each data vertical (`https://example.com/import/blobs`, `https://example.com/import/social-posts`). | 
|  | 7 | + | 
|  | 8 | +### HTTP Interface | 
|  | 9 | + | 
|  | 10 | +Data will be POSTed to your endpoint with a `Content-Type` of either `application/json` for basic data-types, or `multipart/related` for file-based data-types. See below for how to interpret each of these. | 
|  | 11 | +Your endpoint should return a status 201 if the resource has been created, 40x for errors caused by the Importer, or 50x for service errors. | 
|  | 12 | + | 
|  | 13 | +#### Basic data types | 
|  | 14 | + | 
|  | 15 | +For basic data-types the Importer will send a POST request with a `Content-Type: application/json` header. The body of the POST request will be a UTF-8 encoded JSON payload conforming to the relevant data-type schema detailed in the [Schemas](#schemas) section. | 
|  | 16 | + | 
|  | 17 | +For example, below is a full request of a SOCIAL_POST item (JSON formatted for readability) | 
|  | 18 | + | 
|  | 19 | +```http | 
|  | 20 | +POST /import/social-posts HTTP/1.1 | 
|  | 21 | +Content-Type: application/json | 
|  | 22 | +Content-Length: 510 | 
|  | 23 | +Host: localhost:8080 | 
|  | 24 | +Connection: Keep-Alive | 
|  | 25 | +Accept-Encoding: gzip | 
|  | 26 | +User-Agent: okhttp/3.9.1 | 
|  | 27 | +
 | 
|  | 28 | +{ | 
|  | 29 | +  "@type": "SocialActivityData", | 
|  | 30 | +  "metadata": { | 
|  | 31 | +    "@type": "SocialActivityMetadata", | 
|  | 32 | +    "actor": { | 
|  | 33 | +      "@type": "SocialActivityActor", | 
|  | 34 | +      "id": "321", | 
|  | 35 | +      "name": "Steve", | 
|  | 36 | +      "url": null | 
|  | 37 | +    } | 
|  | 38 | +  }, | 
|  | 39 | +  "activity": { | 
|  | 40 | +    "@type": "SocialActivityModel", | 
|  | 41 | +    "id": "456", | 
|  | 42 | +    "published": 1731604863.845677, | 
|  | 43 | +    "type": "NOTE", | 
|  | 44 | +    "attachments": [ | 
|  | 45 | +      { | 
|  | 46 | +        "@type": "SocialActivityAttachment", | 
|  | 47 | +        "type": "IMAGE", | 
|  | 48 | +        "url": "foo.com", | 
|  | 49 | +        "name": "Foo", | 
|  | 50 | +        "content": null | 
|  | 51 | +      } | 
|  | 52 | +    ], | 
|  | 53 | +    "location": { | 
|  | 54 | +      "@type": "SocialActivityLocation", | 
|  | 55 | +      "name": "foo", | 
|  | 56 | +      "longitude": 10.0, | 
|  | 57 | +      "latitude": 10.0 | 
|  | 58 | +    }, | 
|  | 59 | +    "title": "Hello world!", | 
|  | 60 | +    "content": "Hi there", | 
|  | 61 | +    "url": null | 
|  | 62 | +  } | 
|  | 63 | +} | 
|  | 64 | +``` | 
|  | 65 | + | 
|  | 66 | +#### File-based data types | 
|  | 67 | + | 
|  | 68 | +To support large files as part of MEDIA, PHOTOS, VIDEOS and BLOBS imports, the request for these data-types is a `multipart/related` request with a `boundary`. \ | 
|  | 69 | +The first part will be an `application/json` payload containing metadata related to the file being importer. \ | 
|  | 70 | +The second part will have its `Content-Type` set to the MIME type of the file, and will be a multi-part stream of the bytes in the file with `boundary` separators. | 
|  | 71 | + | 
|  | 72 | +```http | 
|  | 73 | +POST /import/blobs HTTP/1.1 | 
|  | 74 | +Content-Type: multipart/related; boundary=1581c5eb-05d0-42fe-bfb3-472151f366cd | 
|  | 75 | +Content-Length: 524288384 | 
|  | 76 | +Host: localhost:8080 | 
|  | 77 | +Connection: Keep-Alive | 
|  | 78 | +Accept-Encoding: gzip | 
|  | 79 | +User-Agent: okhttp/3.9.1 | 
|  | 80 | +
 | 
|  | 81 | +--1581c5eb-05d0-42fe-bfb3-472151f366cd | 
|  | 82 | +Content-Type: application/json | 
|  | 83 | +Content-Length: 130 | 
|  | 84 | +
 | 
|  | 85 | +{"@type":"BlobbyFileData","folder":"/root/foo","document":{"name":"bar.mp4","dateModified":"2020-02-01","encodingFormat":"video/mp4"}} | 
|  | 86 | +--1581c5eb-05d0-42fe-bfb3-472151f366cd | 
|  | 87 | +Content-Type: video/mp4 | 
|  | 88 | +Content-Length: 524288000 | 
|  | 89 | +
 | 
|  | 90 | +...524288000 bytes follow | 
|  | 91 | +--1581c5eb-05d0-42fe-bfb3-472151f366cd-- | 
|  | 92 | +``` | 
|  | 93 | + | 
|  | 94 | +## Schemas | 
|  | 95 | + | 
|  | 96 | +```json | 
|  | 97 | +{ | 
|  | 98 | +    "TODO": "schema" | 
|  | 99 | +} | 
|  | 100 | +``` | 
0 commit comments