Overview.html 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227
  1. <!DOCTYPE html>
  2. <html dir="ltr">
  3. <head>
  4. <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  5. <title>App/Databases/Overview - XOWA</title>
  6. <link rel="shortcut icon" href="https://gnosygnu.github.io/xowa/xowa_logo.png" />
  7. <link rel="stylesheet" href="https://gnosygnu.github.io/xowa/xowa_common.css" type="text/css">
  8. </head>
  9. <body class="mediawiki ltr sitedir-ltr ns-0 ns-subject skin-vector action-submit vector-animateLayout" spellcheck="false">
  10. <div id="mw-page-base" class="noprint"></div>
  11. <div id="mw-head-base" class="noprint"></div>
  12. <div id="content" class="mw-body">
  13. <h1 id="firstHeading" class="firstHeading"><span>App/Databases/Overview</span></h1>
  14. <div id="bodyContent" class="mw-body-content">
  15. <div id="siteSub">From XOWA: the free, open-source, offline wiki application</div>
  16. <div id="contentSub"></div>
  17. <div id="mw-content-text" lang="en" dir="ltr" class="mw-content-ltr">
  18. <p>
  19. XOWA stores wiki data in SQLite databases. These databases can be classified as either Wikitext or HTML
  20. </p>
  21. <div id="toc" class="toc">
  22. <div id="toctitle" class="toctitle">
  23. <h2>
  24. Contents
  25. </h2>
  26. </div>
  27. <ul>
  28. <li class="toclevel-1 tocsection-1">
  29. <a href="#SQLite"><span class="tocnumber">1</span> <span class="toctext">SQLite</span></a>
  30. </li>
  31. <li class="toclevel-1 tocsection-2">
  32. <a href="#XOWA_Database_sets:_Wikitext_vs_HTML"><span class="tocnumber">2</span> <span class="toctext">XOWA Database sets: Wikitext vs HTML</span></a>
  33. <ul>
  34. <li class="toclevel-2 tocsection-3">
  35. <a href="#Wikitext"><span class="tocnumber">2.1</span> <span class="toctext">Wikitext</span></a>
  36. </li>
  37. <li class="toclevel-2 tocsection-4">
  38. <a href="#HTML"><span class="tocnumber">2.2</span> <span class="toctext">HTML</span></a>
  39. </li>
  40. </ul>
  41. </li>
  42. </ul>
  43. </div>
  44. <h2>
  45. <span class="mw-headline" id="SQLite">SQLite</span>
  46. </h2>
  47. <p>
  48. XOWA uses SQLite database files to store its data.
  49. </p>
  50. <ul>
  51. <li>
  52. Files have an extension of either ".xowa" or ".sqlite3". Note that ".xowa" files are just plain old sqlite3 databases with a renamed extension.
  53. </li>
  54. <li>
  55. Files are limited to 2 GB. Note that SQLite can easily handle files greater than 100 GB. The 2 GB limit is imposed for the following two constraints:
  56. <ul>
  57. <li>
  58. <b>2 GB limit for SQLite files on Android</b>. I've not found any documentation that states this, but opening a file larger than 2 GB file always fails.
  59. </li>
  60. <li>
  61. <b>4 GB limit for FAT32 on flash cards</b>. FAT32 is an old file-system format. However, it is the only format that is supported natively by all major operating systems. I install a set of wikis on a 200 GB FAT32-formatted microSD card and carry it around from machine to machine
  62. </li>
  63. </ul>
  64. </li>
  65. </ul>
  66. <p>
  67. Various programs can be used to read SQLite3.
  68. </p>
  69. <ul>
  70. <li>
  71. <b>sqlite3 command shell</b>: This is the official shell for SQLite. Instructions are available here: the <a href="https://www.sqlite.org/cli.html" rel="nofollow" class="external free">https://www.sqlite.org/cli.html</a> . Download is available here: <a href="https://www.sqlite.org/download.html" rel="nofollow" class="external free">https://www.sqlite.org/download.html</a>
  72. </li>
  73. <li>
  74. <b>SQLite browser</b>. This is a free, open-source GUI that can be useful for those who don't want just a command-line shell. See: <a href="http://sqlitebrowser.org/" rel="nofollow" class="external free">http://sqlitebrowser.org/</a>
  75. </li>
  76. </ul>
  77. <h2>
  78. <span class="mw-headline" id="XOWA_Database_sets:_Wikitext_vs_HTML">XOWA Database sets: Wikitext vs HTML</span>
  79. </h2>
  80. <p>
  81. There are two types of database sets for XOWA: Wikitext and HTML
  82. </p>
  83. <h3>
  84. <span class="mw-headline" id="Wikitext">Wikitext</span>
  85. </h3>
  86. <ul>
  87. <li>
  88. Wikitext is a custom language by MediaWiki to generate HTML. For example, two apostrophes are used for italics: <code>''italics''</code>
  89. </li>
  90. <li>
  91. Wikitext databases will have <code>-text-</code> in the file name. For example, <code>en.wikipedia.org-text-ns.000.xowa</code>
  92. </li>
  93. <li>
  94. Users can generates this Wikitext set by using either <a href="/wiki/Dashboard/Import/Online" id="xolnki_2" title="Dashboard/Import/Online">Dashboard/Import/Online</a> or <a href="/wiki/Dashboard/Import/Offline" id="xolnki_3" title="Dashboard/Import/Offline">Dashboard/Import/Offline</a>
  95. </li>
  96. <li>
  97. Wikitext databases have been supported since the very start of XOWA. As such, they have a full set of functionality
  98. </li>
  99. <li>
  100. However, Wikitext databases are slower. They need to take the Wikitext and convert it to HTML. As such, Wikitext databases don't work for Android, nor for other low-spec machines like Raspberry Pi
  101. </li>
  102. </ul>
  103. <h3>
  104. <span class="mw-headline" id="HTML">HTML</span>
  105. </h3>
  106. <ul>
  107. <li>
  108. HTML is the raw text as used by a web browser. For example, the following is used for italics: <code>&lt;i&gt;italics&lt;/i&gt;</code>
  109. </li>
  110. <li>
  111. HTML databases will have <code>-html-</code> in the file name. For example, <code>en.wikipedia.org-html.xowa</code>
  112. </li>
  113. <li>
  114. Users can generate this HTML database using <a href="/wiki/Dev/Command-line/Thumbs" id="xolnki_4" title="Dev/Command-line/Thumbs">Dev/Command-line/Thumbs</a>. However, for most users, they will just download the already-generated HTML databases from <a href="http://archive.org" rel="nofollow" class="external free">http://archive.org</a> through <a href="/wiki/Special:XowaDownloadCentral" id="xolnki_5" title="Special:XowaDownloadCentral">Special:XowaDownloadCentral</a>
  115. </li>
  116. <li>
  117. HTML databases have only been supported recently (as of 2016-06). As such, they are missing a small percentage of functionality. The most significant piece of that functionality is Popups, but there may be other features.
  118. </li>
  119. <li>
  120. HTML databases are fast, as they don't need to convert Wikitext to HTML. As such, HTML databases are used for Android and recommended for Raspberry Pi
  121. </li>
  122. </ul>
  123. </div>
  124. </div>
  125. </div>
  126. <div id="mw-head" class="noprint">
  127. <div id="left-navigation">
  128. <div id="p-namespaces" class="vectorTabs">
  129. <h3>Namespaces</h3>
  130. <ul>
  131. <li id="ca-nstab-main" class="selected"><span><a id="ca-nstab-main-href" href="index.html">Page</a></span></li>
  132. </ul>
  133. </div>
  134. </div>
  135. </div>
  136. <div id='mw-panel' class='noprint'>
  137. <div id='p-logo'>
  138. <a style="background-image: url(https://gnosygnu.github.io/xowa/xowa_logo.png);" href="http://xowa.org/" title="Visit the main page"></a>
  139. </div>
  140. <div class="portal" id='xowa-portal-home'>
  141. <h3>XOWA</h3>
  142. <div class="body">
  143. <ul>
  144. <li><a href="http://xowa.org/index.html" title='Visit the main page'>Main page</a></li>
  145. <li><a href="http://xowa.org/screenshots.html" title='See screenshots of XOWA'>Screenshots</a></li>
  146. <li><a href="https://www.youtube.com/watch?v=q0qbXYXEH6M" title="See a video of XOWA Desktop in action">Video</a></li>
  147. <li><a href="http://xowa.org/home/wiki/Help/Download_XOWA.html" title='Download the XOWA application'>Download XOWA</a></li>
  148. <li><a href="http://xowa.org/home/wiki/Dashboard/Image_databases.html" title='Download offline wikis and image databases'>Download wikis</a></li>
  149. </ul>
  150. </div>
  151. </div>
  152. <div class="portal" id='xowa-portal-started'>
  153. <h3>Getting started</h3>
  154. <div class="body">
  155. <ul>
  156. <li><a href="http://xowa.org/home/wiki/App/Setup/System_requirements.html" title='Get XOWA&apos;s system requirements'>Requirements</a></li>
  157. <li><a href="http://xowa.org/home/wiki/App/Setup/Installation.html" title='Get instructions for installing XOWA'>Installation</a></li>
  158. <li><a href="http://xowa.org/home/wiki/App/Import/Simple_Wikipedia.html" title='Learn how to set up Simple Wikipedia'>Simple Wikipedia</a></li>
  159. <li><a href="http://xowa.org/home/wiki/App/Import/English_Wikipedia.html" title='Learn how to set up English Wikipedia'>English Wikipedia</a></li>
  160. <li><a href="http://xowa.org/home/wiki/App/Import/Other_wikis.html" title='Learn how to set up other Wikipedias'>Other Wikipedias</a></li>
  161. </ul>
  162. </div>
  163. </div>
  164. <div class="portal" id='xowa-portal-android'>
  165. <h3>Android</h3>
  166. <div class="body">
  167. <ul>
  168. <li><a href="http://xowa.org/home/wiki/Android/Setup.html" title='Setup XOWA on your Android device'>Setup</a></li>
  169. <li><a href="https://www.youtube.com/watch?v=jsMTBxGweUw" title="See a video of XOWA Android in action">Video</a></li>
  170. </ul>
  171. </div>
  172. </div>
  173. <div class="portal" id='xowa-portal-help'>
  174. <h3>Help</h3>
  175. <div class="body">
  176. <ul>
  177. <li><a href="http://xowa.org/home/wiki/Help/About.html" title='Get more information about XOWA'>About</a></li>
  178. <li><a href="http://xowa.org/home/wiki/Help/Contents.html" title='View a list of help topics'>Contents</a></li>
  179. <li><a href="http://xowa.org/home/wiki/Help/Media.html" title='Read what others have written about XOWA'>Media</a></li>
  180. <li><a href="http://xowa.org/home/wiki/Help/Feedback.html" title='Questions? Comments? Leave feedback for XOWA'>Feedback</a></li>
  181. </ul>
  182. </div>
  183. </div>
  184. <div class="portal" id='xowa-portal-blog'>
  185. <h3>Blog</h3>
  186. <div class="body">
  187. <ul>
  188. <li><a href="http://xowa.org/home/wiki/Blog.html" title='Follow XOWA''s development process'>Current</a></li>
  189. </ul>
  190. </div>
  191. </div>
  192. <div class="portal" id='xowa-portal-links'>
  193. <h3>Links</h3>
  194. <div class="body">
  195. <ul>
  196. <li><a href="http://dumps.wikimedia.org/backup-index.html" title="Get wiki datababase dumps directly from Wikimedia">Wikimedia dumps</a></li>
  197. <li><a href="https://archive.org/search.php?query=xowa" title="Search archive.org for XOWA files">XOWA @ archive.org</a></li>
  198. <li><a href="http://en.wikipedia.org" title="Visit Wikipedia (and compare to XOWA!)">English Wikipedia</a></li>
  199. </ul>
  200. </div>
  201. </div>
  202. <div class="portal" id='xowa-portal-donate'>
  203. <h3>Donate</h3>
  204. <div class="body">
  205. <ul>
  206. <li><a href="https://archive.org/donate/index.php" title="Support archive.org!">archive.org</a></li><!-- listed first due to recent fire damages: http://blog.archive.org/2013/11/06/scanning-center-fire-please-help-rebuild/ -->
  207. <li><a href="https://donate.wikimedia.org/wiki/Special:FundraiserRedirector" title="Support Wikipedia!">Wikipedia</a></li>
  208. <li><a href="http://xowa.org/home/wiki/Help/Donate.html" title="Support XOWA!">XOWA</a></li>
  209. </ul>
  210. </div>
  211. </div>
  212. </div>
  213. </body>
  214. </html>