Database.html 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391
  1. <!DOCTYPE html>
  2. <html dir="ltr">
  3. <head>
  4. <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  5. <title>Dev/Design/Database - XOWA</title>
  6. <link rel="shortcut icon" href="https://gnosygnu.github.io/xowa/xowa_logo.png" />
  7. <link rel="stylesheet" href="https://gnosygnu.github.io/xowa/xowa_common.css" type="text/css">
  8. </head>
  9. <body class="mediawiki ltr sitedir-ltr ns-0 ns-subject skin-vector action-submit vector-animateLayout" spellcheck="false">
  10. <div id="mw-page-base" class="noprint"></div>
  11. <div id="mw-head-base" class="noprint"></div>
  12. <div id="content" class="mw-body">
  13. <h1 id="firstHeading" class="firstHeading"><span>Dev/Design/Database</span></h1>
  14. <div id="bodyContent" class="mw-body-content">
  15. <div id="siteSub">From XOWA: the free, open-source, offline wiki application</div>
  16. <div id="contentSub"></div>
  17. <div id="mw-content-text" lang="en" dir="ltr" class="mw-content-ltr">
  18. <div id="toc" class="toc">
  19. <div id="toctitle">
  20. <h2>
  21. Contents
  22. </h2>
  23. </div>
  24. <ul>
  25. <li class="toclevel-1 tocsection-1">
  26. <a href="#Overview"><span class="tocnumber">1</span> <span class="toctext">Overview</span></a>
  27. </li>
  28. <li class="toclevel-1 tocsection-2">
  29. <a href="#Details"><span class="tocnumber">2</span> <span class="toctext">Details</span></a>
  30. </li>
  31. <li class="toclevel-1 tocsection-3">
  32. <a href="#Options"><span class="tocnumber">3</span> <span class="toctext">Options</span></a>
  33. </li>
  34. <li class="toclevel-1 tocsection-4">
  35. <a href="#Multi-file_layout"><span class="tocnumber">4</span> <span class="toctext">Multi-file layout</span></a>
  36. </li>
  37. <li class="toclevel-1 tocsection-5">
  38. <a href="#Version_compatibility"><span class="tocnumber">5</span> <span class="toctext">Version compatibility</span></a>
  39. </li>
  40. </ul>
  41. </div>
  42. <h2>
  43. <span class="mw-headline" id="Overview">Overview</span>
  44. </h2>
  45. <p>
  46. As of v2.4.1.1, XOWA has changed the database layout to be more simple and portable. Regular users should not notice any difference. Users who move XOWA wikis between computer systems will notice some benefits.
  47. </p>
  48. <h2>
  49. <span class="mw-headline" id="Details">Details</span>
  50. </h2>
  51. <ul>
  52. <li>
  53. <b>.xowa</b>: XOWA databases now have the extension <code>.xowa</code> instead of <code>.sqlite3</code>.
  54. <ul>
  55. <li>
  56. For example, importing Simple Wikipedia now generates a file called <code>simple.wikipedia.org-text.xowa</code> instead of <code>simple.wikipedia.org.000.sqlite3</code>
  57. </li>
  58. <li>
  59. Note that the underlying file is still a sqlite3 database. The extension has just been renamed from <code>.sqlite3</code> to <code>.xowa</code>.
  60. </li>
  61. <li>
  62. Note that all new wikis uploaded to archive.org will be in .xowa format. The same also goes for new torrents.
  63. </li>
  64. </ul>
  65. </li>
  66. <li>
  67. <b>Backward compatible</b>: XOWA v2.4.1.1 and higher opens both <code>.xowa</code> wikis and <code>.sqlite3</code> wikis. There is no need to upgrade to the new wikis unless you're planning on using some of the features below. However, users are encouraged to generate new wikis with the new format.
  68. </li>
  69. <li>
  70. <b>One file for most wikis</b>: Small wikis will default to one file instead of several.
  71. <ul>
  72. <li>
  73. For example, importing simple.wikipedia.org will generate 1 file called <code>simple.wikipedia.org-text.xowa</code>. In contrast, it used to generate 5 files: <code>simple.wikipedia.org.000.sqlite3</code> up to <code>simple.wikipedia.org.004.sqlite3</code>
  74. </li>
  75. </ul>
  76. </li>
  77. <li>
  78. <b>Multiple 'detachable' files for large wikis</b>: Larger wikis will still generate multiple files. See the Multi-file layout section below for more detail.
  79. </li>
  80. <li>
  81. <b>Databases in same directory</b>: All database files are now under <code>/xowa/wiki/wiki_name</code>. File databases no longer have to go to <code>/xowa/file/wiki_name</code>. The latter was a point of confusion for new users, as well as an inconvenience when trying to move wikis to new machines.
  82. </li>
  83. </ul>
  84. <h2>
  85. <span class="mw-headline" id="Options">Options</span>
  86. </h2>
  87. <p>
  88. General behavior for "single" vs "multiple" file import can be customized at <a href="http://xowa.org/wiki/home/page/Options/Import.html" id="xolnki_2" title="Options/Import" class="xowa-visited">Options/Import</a> under Database Layout
  89. </p>
  90. <h2>
  91. <span class="mw-headline" id="Multi-file_layout">Multi-file layout</span>
  92. </h2>
  93. <p>
  94. Large wikis will still generate multiple files. This includes English Wikipedia, German Wikipedia as well as about a dozen others. The primary reasons are as follows:
  95. </p>
  96. <ul>
  97. <li>
  98. <b>FAT32 4 GB limit</b>: FAT32 supports a maximum file size of 4 GB. Although modern operating can support files greater than 4 GB, most flash cards still use a FAT32 file-system. In addition, large files are not supported in other formats (for example, DVDs, archive.org with a 10 GB limit, etc.)
  99. </li>
  100. <li>
  101. <b>Detachable databases</b>: Wikis will still work even if individual databases are deleted or removed. For example, if you don't plan to read the articles in the Wikipedia namespace and want to save 2.6 GB of disk space then you can simply remove the file called "en.wikipedia.org-text-ns.004.xowa".
  102. </li>
  103. </ul>
  104. <p>
  105. Note that detached databases can be reattached simply by moving the file back into its proper location. For example, consider the following:
  106. </p>
  107. <ul>
  108. <li>
  109. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-xtn.category.core.xowa</span> is deleted.
  110. <ul>
  111. <li>
  112. Categories will no longer show, but the wiki functions as normal
  113. </li>
  114. </ul>
  115. </li>
  116. <li>
  117. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-xtn.category.core.xowa</span> is restored.
  118. <ul>
  119. <li>
  120. Categories will now work again.
  121. </li>
  122. </ul>
  123. </li>
  124. </ul>
  125. <p>
  126. The following table provides more information on what happens when a database is detached:
  127. </p>
  128. <table class='wikitable'>
  129. <tr>
  130. <th>
  131. file-name suffix
  132. </th>
  133. <th>
  134. example
  135. </th>
  136. <th>
  137. effect if removed
  138. </th>
  139. </tr>
  140. <tr>
  141. <td>
  142. core
  143. </td>
  144. <td>
  145. en.wikipedia.org-core.xowa
  146. </td>
  147. <td>
  148. Core file: wiki will not work
  149. </td>
  150. </tr>
  151. <tr>
  152. <td>
  153. text-ns.***-db.***
  154. </td>
  155. <td>
  156. en.wikipedia.org-text-ns.012.xowa
  157. </td>
  158. <td>
  159. Articles in the namespace will not be available. See <a href="http://en.wikipedia.org/wiki/Wikipedia:Namespace" rel="nofollow" class="external free">http://en.wikipedia.org/wiki/Wikipedia:Namespace</a> For example, ns.012 is for "Help:" articles
  160. </td>
  161. </tr>
  162. <tr>
  163. <td>
  164. xtn.category
  165. </td>
  166. <td>
  167. en.wikipedia.org-xtn.category.xowa
  168. </td>
  169. <td>
  170. Categories will not show up at the bottom of the article
  171. </td>
  172. </tr>
  173. <tr>
  174. <td>
  175. xtn.index
  176. </td>
  177. <td>
  178. en.wikipedia.org-xtn.index.xowa
  179. </td>
  180. <td>
  181. "Special:Search" will not work; EX: en.wikipedia.org/wiki/Special:Search/Earth?search=Earth&amp;fulltext=y
  182. </td>
  183. </tr>
  184. <tr>
  185. <td>
  186. wbase
  187. </td>
  188. <td>
  189. www.wikidata.org-wbase.xowa
  190. </td>
  191. <td>
  192. Core file: Wikidata will not work.
  193. </td>
  194. </tr>
  195. <tr>
  196. <td>
  197. file-core
  198. </td>
  199. <td>
  200. en.wikipedia.org-file-core.xowa
  201. </td>
  202. <td>
  203. Wiki will not show images
  204. </td>
  205. </tr>
  206. <tr>
  207. <td>
  208. file-data
  209. </td>
  210. <td>
  211. en.wikipedia.org-file-data.001.xowa
  212. </td>
  213. <td>
  214. Images in that database will not show
  215. </td>
  216. </tr>
  217. </table>
  218. <h2>
  219. <span class="mw-headline" id="Version_compatibility">Version compatibility</span>
  220. </h2>
  221. <p>
  222. The formats for wiks and files are interchangable between old and new formats. A user may keep their existing <b>old</b> file databases and still generate <b>new</b> wikis whenever they want to.
  223. </p>
  224. <p>
  225. For example, consider English Wikipedia. All of the following combinations of <b>old</b> and <b>new</b> should work:
  226. </p>
  227. <table class="wikitable">
  228. <tr>
  229. <th>
  230. wiki type
  231. </th>
  232. <th>
  233. file type
  234. </th>
  235. <th>
  236. wiki path example
  237. </th>
  238. <th>
  239. file path example
  240. </th>
  241. </tr>
  242. <tr>
  243. <td>
  244. old
  245. </td>
  246. <td>
  247. old
  248. </td>
  249. <td>
  250. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org.000.sqlite3</span>
  251. </td>
  252. <td>
  253. <span class='path'>C:\xowa\file\en.wikipedia.org\fsdb.main\fsdb.abc.sqlite3</span>
  254. </td>
  255. </tr>
  256. <tr>
  257. <td>
  258. new
  259. </td>
  260. <td>
  261. old
  262. </td>
  263. <td>
  264. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-core.xowa</span>
  265. </td>
  266. <td>
  267. <span class='path'>C:\xowa\file\en.wikipedia.org\fsdb.main\fsdb.abc.sqlite3</span>
  268. </td>
  269. </tr>
  270. <tr>
  271. <td>
  272. old
  273. </td>
  274. <td>
  275. new
  276. </td>
  277. <td>
  278. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org.000.sqlite3</span>
  279. </td>
  280. <td>
  281. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-file-core.xowa</span>
  282. </td>
  283. </tr>
  284. <tr>
  285. <td>
  286. new
  287. </td>
  288. <td>
  289. new
  290. </td>
  291. <td>
  292. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-core.xowa</span>
  293. </td>
  294. <td>
  295. <span class='path'>C:\xowa\wiki\en.wikipedia.org\en.wikipedia.org-file-core.xowa</span>
  296. </td>
  297. </tr>
  298. </table>
  299. </div>
  300. </div>
  301. </div>
  302. <div id="mw-head" class="noprint">
  303. <div id="left-navigation">
  304. <div id="p-namespaces" class="vectorTabs">
  305. <h3>Namespaces</h3>
  306. <ul>
  307. <li id="ca-nstab-main" class="selected"><span><a id="ca-nstab-main-href" href="index.html">Page</a></span></li>
  308. </ul>
  309. </div>
  310. </div>
  311. </div>
  312. <div id='mw-panel' class='noprint'>
  313. <div id='p-logo'>
  314. <a style="background-image: url(https://gnosygnu.github.io/xowa/xowa_logo.png);" href="http://xowa.org/" title="Visit the main page"></a>
  315. </div>
  316. <div class="portal" id='xowa-portal-home'>
  317. <h3>XOWA</h3>
  318. <div class="body">
  319. <ul>
  320. <li><a href="http://xowa.org/index.html" title='Visit the main page'>Main page</a></li>
  321. <li><a href="http://xowa.org/screenshots.html" title='See screenshots of XOWA'>Screenshots</a></li>
  322. <li><a href="http://xowa.org/wiki/home/page/Help/Download_XOWA.html" title='Download the XOWA application'>Download XOWA</a></li>
  323. <li><a href="http://xowa.org/wiki/home/page/Dashboard/Image_databases.html" title='Download offline wikis and image databases'>Download wikis</a></li>
  324. </ul>
  325. </div>
  326. </div>
  327. <div class="portal" id='xowa-portal-stargin'>
  328. <h3>Getting started</h3>
  329. <div class="body">
  330. <ul>
  331. <li><a href="http://xowa.org/wiki/home/page/App/Setup/System_requirements.html" title='Get XOWA&apos;s system requirements'>Requirements</a></li>
  332. <li><a href="http://xowa.org/wiki/home/page/App/Setup/Installation.html" title='Get instructions for installing XOWA'>Installation</a></li>
  333. <li><a href="http://xowa.org/wiki/home/page/App/Import/Simple_Wikipedia.html" title='Learn how to set up Simple Wikipedia'>Simple Wikipedia</a></li>
  334. <li><a href="http://xowa.org/wiki/home/page/App/Import/English_Wikipedia.html" title='Learn how to set up English Wikipedia'>English Wikipedia</a></li>
  335. <li><a href="http://xowa.org/wiki/home/page/App/Import/Other_wikis.html" title='Learn how to set up other Wikipedias'>Other Wikipedias</a></li>
  336. </ul>
  337. </div>
  338. </div>
  339. <div class="portal" id='xowa-portal-help'>
  340. <h3>Help</h3>
  341. <div class="body">
  342. <ul>
  343. <li><a href="http://xowa.org/wiki/home/page/Help/About.html" title='Get more information about XOWA'>About</a></li>
  344. <li><a href="http://xowa.org/wiki/home/page/Help/Contents.html" title='View a list of help topics'>Contents</a></li>
  345. <li><a href="http://xowa.org/wiki/home/page/Help/Media.html" title='Read what others have written about XOWA'>Media</a></li>
  346. <li><a href="http://xowa.org/wiki/home/page/Help/Feedback.html" title='Questions? Comments? Leave feedback for XOWA'>Feedback</a></li>
  347. </ul>
  348. </div>
  349. </div>
  350. <div class="portal" id='xowa-portal-blog'>
  351. <h3>Blog</h3>
  352. <div class="body">
  353. <ul>
  354. <li><a href="http://xowa.org/wiki/home/page/Blog.html" title='Follow XOWA''s development process'>Current</a></li>
  355. </ul>
  356. </div>
  357. </div>
  358. <div class="portal" id='xowa-portal-links'>
  359. <h3>Links</h3>
  360. <div class="body">
  361. <ul>
  362. <li><a href="http://dumps.wikimedia.org/backup-index.html" title="Get wiki datababase dumps directly from Wikimedia">Wikimedia dumps</a></li>
  363. <li><a href="https://archive.org/search.php?query=xowa" title="Search archive.org for XOWA files">XOWA @ archive.org</a></li>
  364. <li><a href="http://en.wikipedia.org" title="Visit Wikipedia (and compare to XOWA!)">English Wikipedia</a></li>
  365. </ul>
  366. </div>
  367. </div>
  368. <div class="portal" id='xowa-portal-donate'>
  369. <h3>Donate</h3>
  370. <div class="body">
  371. <ul>
  372. <li><a href="https://archive.org/donate/index.php" title="Support archive.org!">archive.org</a></li><!-- listed first due to recent fire damages: http://blog.archive.org/2013/11/06/scanning-center-fire-please-help-rebuild/ -->
  373. <li><a href="https://donate.wikimedia.org/wiki/Special:FundraiserRedirector" title="Support Wikipedia!">Wikipedia</a></li>
  374. <!-- <li><a href="" title="Support XOWA! (but only after you've supported archive.org and Wikipedia)">XOWA</a></li> -->
  375. </ul>
  376. </div>
  377. </div>
  378. </div>
  379. </body>
  380. </html>