Skip to content

Commit 312862b

Browse files
committed
Add README with draft docs for generic importers
1 parent 80c55cb commit 312862b

File tree

1 file changed

+104
-0
lines changed
  • extensions/data-transfer/portability-data-transfer-generic

1 file changed

+104
-0
lines changed
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Generic Importers
2+
3+
## Implementing a Server
4+
5+
To consume data from a generic importer, create a HTTP service in the language of your choice.
6+
The service should accept POST requests on a given base URL, and kebab-case sub-path specific to each data vertical (`https://example.com/import/blobs`, `https://example.com/import/social-posts`).
7+
8+
### HTTP Interface
9+
10+
Data will be POSTed to your endpoint with a `Content-Type` of either `application/json` for basic data-types, or `multipart/related` for file-based data-types. See below for how to interpret each of these.
11+
Your endpoint should return a status 201 if the resource has been created, 40x for errors caused by the Importer, or 50x for service errors.
12+
13+
#### Basic data types
14+
15+
For basic data-types the Importer will send a POST request with a `Content-Type: application/json` header. The body of the POST request will be a UTF-8 encoded JSON payload conforming to the relevant data-type schema detailed in the [Schemas](#schemas) section.
16+
17+
For example, below is a full request of a SOCIAL_POST item (JSON formatted for readability)
18+
19+
```http
20+
POST /import/social-posts HTTP/1.1
21+
Content-Type: application/json
22+
Content-Length: 510
23+
Host: localhost:8080
24+
Connection: Keep-Alive
25+
Accept-Encoding: gzip
26+
User-Agent: okhttp/3.9.1
27+
28+
{
29+
"@type": "SocialActivityData",
30+
"metadata": {
31+
"@type": "SocialActivityMetadata",
32+
"actor": {
33+
"@type": "SocialActivityActor",
34+
"id": "321",
35+
"name": "Steve",
36+
"url": null
37+
}
38+
},
39+
"activity": {
40+
"@type": "SocialActivityModel",
41+
"id": "456",
42+
"published": 1731604863.845677,
43+
"type": "NOTE",
44+
"attachments": [
45+
{
46+
"@type": "SocialActivityAttachment",
47+
"type": "IMAGE",
48+
"url": "foo.com",
49+
"name": "Foo",
50+
"content": null
51+
}
52+
],
53+
"location": {
54+
"@type": "SocialActivityLocation",
55+
"name": "foo",
56+
"longitude": 10.0,
57+
"latitude": 10.0
58+
},
59+
"title": "Hello world!",
60+
"content": "Hi there",
61+
"url": null
62+
}
63+
}
64+
```
65+
66+
#### File-based data types
67+
68+
To support large files as part of MEDIA, PHOTOS, VIDEOS and BLOBS imports, the request for these data-types is a `multipart/related` request with a `boundary`. \
69+
The first part will be an `application/json` payload containing metadata related to the file being importer. \
70+
The second part will have its `Content-Type` set to the MIME type of the file, and will be a multi-part stream of the bytes in the file with `boundary` separators.
71+
72+
```http
73+
POST /import/blobs HTTP/1.1
74+
Content-Type: multipart/related; boundary=1581c5eb-05d0-42fe-bfb3-472151f366cd
75+
Content-Length: 524288384
76+
Host: localhost:8080
77+
Connection: Keep-Alive
78+
Accept-Encoding: gzip
79+
User-Agent: okhttp/3.9.1
80+
81+
--1581c5eb-05d0-42fe-bfb3-472151f366cd
82+
Content-Type: application/json
83+
Content-Length: 130
84+
85+
{"@type":"BlobbyFileData","folder":"/root/foo","document":{"name":"bar.mp4","dateModified":"2020-02-01","encodingFormat":"video/mp4"}}
86+
--1581c5eb-05d0-42fe-bfb3-472151f366cd
87+
Content-Type: video/mp4
88+
Content-Length: 524288000
89+
90+
...524288000 bytes follow
91+
--1581c5eb-05d0-42fe-bfb3-472151f366cd--
92+
```
93+
94+
> [!NOTE]
95+
> Endpoints supporting file-based data-types may also receive basic data-types, for example the `/blobs` endpoint should
96+
> support receiving folders encoded as `application/json` POST data.
97+
98+
## Schemas
99+
100+
```json
101+
{
102+
"TODO": "schema"
103+
}
104+
```

0 commit comments

Comments
 (0)