Skip to content

[Wikibase] Working with JSON dump

Chen edited this page Jan 13, 2018 · 4 revisions

In some cases, you might wish to import & analyze Wikibase entities from serialized JSON. For example, if you are processing the JSON dump generated by dumpJson.php (extensions/Wikibase/repo/maintenance/dumpJson.php), which is actually an array of serialized entities, now it is possible to do so with SerializableEntity class.

The structure of JSON dump

First of all, suppose you are at the root of your MediaWiki installation, you may dump your Wikibase entities with the following command

php ./extensions/Wikibase/repo/maintenance/dumpJson.php --no-cache --output=entities.json

Now entities.json contains a huge JSON array of dumped wikibase entities, one line per minified JSON of a Wikibase item. Each array item has the structure as follows

{
    "type": "item",
    "sitelinks": [
// ...
    ],
    "descriptions": {
        "en": {
            "language": "en",
            "value": "totality of space and all matter and radiation in it,  including planets, galaxies, light, and us; may include their properties such as energy; may include time/spacetime"
        },
        "zh": {
            "language": "zh",
            "value": "一切空间、时间、物质和能量构成的总体"
        },
// ...
    },
    "id": "Q2",
    "claims": {
        "P2": [{
            "type": "statement",
            "references": [],
            "mainsnak": {
                "snaktype": "value",
                "property": "P2",
                "datavalue": {
                    "value": "Q1",
                    "type": "string"
                },
                "datatype": "external-id"
            },
            "qualifiers": [],
            "id": "Q2$997A7A7B-8737-49B6-9386-BD934CE9E2A7",
            "rank": "normal"
        }],
        "P3": [{
            "type": "statement",
            "references": [{
                "hash": "0e556569b6638a2a8a6ee29edef2644b2fc29c15",
                "snaks-order": ["P2"],
                "snaks": {
                    "P2": [{
                        "snaktype": "value",
                        "property": "P2",
                        "datavalue": {
                            "value": "Q1$8983b0ea-4a9c-0902-c0db-785db33f767c",
                            "type": "string"
                        },
                        "datatype": "external-id"
                    }]
                }
            }],
            "mainsnak": {
                "snaktype": "somevalue",
                "property": "P3",
                "datatype": "wikibase-item"
            },
            "qualifiers": [],
            "id": "Q2$47BA934E-9A36-42C5-8767-C4D8D6A3F333",
            "rank": "normal"
        }],
// ...
    },
    "aliases": {
        "en": [{
            "language": "en",
            "value": "Our Universe"
        }, {
            "language": "en",
            "value": "The Universe"
        }, {
            "language": "en",
            "value": "Universe (Ours)"
        }, {
            "language": "en",
            "value": "The Cosmos"
        }, {
            "language": "en",
            "value": "cosmos"
        }]
    },
    "labels": {
        "en": {
            "language": "en",
            "value": "Universe"
        },
        "zh": {
            "language": "zh",
            "value": "宇宙"
        },
// ...
    }
}

[TBC]

Clone this wiki locally