User:AKhatun/Wikidata Subgraph Analysis
What are subgraphs?
Wikidata contains all kinds of data from various aspects of knowledge. All of these data are highly inter-connected, but we can find some patterns. We find subgraphs within Wikidata and find out how large these subgraphs are, how connected they are, and finally how much these subgraphs are used (queried).
In order to find subgraphs, the following steps were taken:
- Consider all items that are instance of
(P31)the same item to be under a subgraph. For example: all items that are
instance ofQ13442814 are part of one subgraph.
- Some subgraphs were merged where it was obvious. For example: all subclasses of astronomical object were considered part of astronomical object as they were all indeed some sort of astronomical object. This method of sublcass merging is not applicable everywhere without manual inspection.
- Some large subgraphs were almost completely part of another subgraph. For example: all items under Review Articles are also instance of scholarly article. In such case, review articles was not considered a separate subgraph.
instance of, Wikidata has
82,919 subgraphs. The distribution of the sizes of these subgraphs has a clear long tail, with very few subgraphs incorporating most items in Wikidata. Subgraph size can be calculated in two ways:
- The number of items it contains
- The number of triples related to the items in a subgraph. This is what we refer as subgraph size from here on.
- Most calculations from here on will take the top 50 subgraphs, which form 85% of Wikidata
- 340 top subgraphs (0.5% of all subgraphs, after merging some) form 90% of Wikidata (91% of all items and 90% of all triples). These subgraphs have >=10,000 items each.
- Rest 99.5% of the subgraphs have <10,000 items each, and together form 10% of Wikidata.
Below is the distribution of the number of items in a subgraphs.
|File:Number of groups vs number of items.png||File:Number of groups vs number of items log.png|
To be more specific,
|Number of subgraphs||Number of items|
|There are||54,602||subgraph(s) with more than||1||item(s)|
Below is the subgraph size comparison of top 340 subgraphs in Wikidata (90%).
Below is the subgraph size comparison of top 50 subgraphs in Wikidata (85%).
Here is an interactive graph showing the comparison of subgraph sizes in terms of item count and triple count: subgraph stats.
Here are some subgraph size visualizations in WDQS:
- Size as percentage of Wikidata each subgraph occupies: query link
- Size as percentage of Wikidata items each subgraph contains: query link
Number of days to recovery Given the current rate of growth, how long would it take wikidata to get back to its original size again if some amount of triples were removed from it? This helps us estimate what to temporarily remove from Wikidata in the siatuation Wikidata backend maxes out. The growth rate of triples is not constant, but considering the growth an approximate straight line, in grafana dashboard, Wikidata grows at a rate of 4.77M triples per day. This rate was calculated from the number of triples at the start and end of a 90-day interval (11/3/21 to 6/6/21). It could be faster or a bit slower than this. This will give us a wide approximation of the number of days we can gain by removing some parts of Wikidata.
|Rank||Subgraph||Subgraph Name||Number of items||% of WD items||Number of triples||% of WD Triples||Number of days to recover|
|7||Q4167410||Wikimedia disambiguation page||1,359,804||1.45||180,124,174||1.37||37.76|
|13||Q13100073||village-level division in China||588,477||0.63||51,615,572||0.39||10.82|
|16||Q13406463||Wikimedia list article||334,939||0.36||33,742,245||0.26||7.07|
|30||Q56436498||village in India||145,824||0.16||15,383,416||0.12||3.23|
|32||Q47150325||calendar day of a given year||189,366||0.2||14,078,486||0.11||2.95|
|34||Q3331189||version, edition, or translation||157,486||0.17||10,997,589||0.08||2.31|
|35||Q18593264||item of collection or exhibition||147,402||0.16||10,732,969||0.08||2.25|
|47||Q59199015||group of stereoisomers||111,599||0.12||5,843,270||0.04||1.23|
|48||Q61443690||branch post office||129,183||0.14||5,313,033||0.04||1.11|