Offline_files.html 9.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194
  1. <!DOCTYPE html>
  2. <html dir="ltr">
  3. <head>
  4. <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  5. <title>Dev/Design/Offline files - XOWA</title>
  6. <link rel="shortcut icon" href="https://gnosygnu.github.io/xowa/xowa_logo.png" />
  7. <link rel="stylesheet" href="https://gnosygnu.github.io/xowa/xowa_common.css" type="text/css">
  8. </head>
  9. <body class="mediawiki ltr sitedir-ltr ns-0 ns-subject skin-vector action-submit vector-animateLayout" spellcheck="false">
  10. <div id="mw-page-base" class="noprint"></div>
  11. <div id="mw-head-base" class="noprint"></div>
  12. <div id="content" class="mw-body">
  13. <h1 id="firstHeading" class="firstHeading"><span>Dev/Design/Offline files</span></h1>
  14. <div id="bodyContent" class="mw-body-content">
  15. <div id="siteSub">From XOWA: the free, open-source, offline wiki application</div>
  16. <div id="contentSub"></div>
  17. <div id="mw-content-text" lang="en" dir="ltr" class="mw-content-ltr">
  18. <table class="metadata plainlinks ambox ambox-delete" style="">
  19. <tr>
  20. <td class="mbox-empty-cell">
  21. </td>
  22. <td class="mbox-text" style="">
  23. <p>
  24. <span class="mbox-text-span">This process is currently in development. The documentation below is provided for reference only. You can run the commands, but please note that the end result will not have any immediate use.</span>
  25. </p>
  26. </td>
  27. </tr>
  28. </table>
  29. <p>
  30. Offline thumbnails will require a lot of time and disk storage. Estimates are provided below in the format of <code>[Space used; Time taken]</code>.
  31. </p>
  32. <p>
  33. Note that Simple Wikipedia will take 24+ hours and 120 GB for the first run.
  34. </p>
  35. <h2>
  36. <span class="mw-headline" id="Commons_process">Commons process</span>
  37. </h2>
  38. <p>
  39. Since most images reside in the commons wiki, commons must be set up. Note that this only needs to be done once for all wikis.
  40. </p>
  41. <ul>
  42. <li>
  43. <code>[ 50 GB; 03 h]</code> Import the Commons wiki with category level 2. See <a href="/wiki/Dashboard/Import/Offline" id="xolnki_2" title="Dashboard/Import/Offline">Dashboard/Import/Offline</a>
  44. </li>
  45. <li>
  46. <code>[ 50 GB; 02 h]</code> Download and unzip commonswiki-latest-image.sql.gz
  47. </li>
  48. <li>
  49. <code>[ 01 GB; 20 m]</code> Run the images command. This will generate a database called oimg_image which will have more data for each file (width, height)
  50. </li>
  51. <li>
  52. <code>[ 15 MB; 40 m]</code> Run the redirect command. This will generate a database called oimg_redirect which will list all redirect targets
  53. </li>
  54. </ul>
  55. <h2>
  56. <span class="mw-headline" id="Wiki_process">Wiki process</span>
  57. </h2>
  58. <p>
  59. These steps must be run for the given wiki. Estimated size / times are provided for Simple Wikipedia.
  60. </p>
  61. <ul>
  62. <li>
  63. <code>[ 01 MB; 01 m]</code> Run the images command.
  64. </li>
  65. <li>
  66. <code>[ 05 MB; 01 m]</code> Run the redirect command.
  67. </li>
  68. <li>
  69. <code>[ 25 MB; 15 m]</code> Run the lnki_temp command. This will generate a database called oimg_lnki which will be the main database. It also creates a table called oimg_lnki_temp which lists every [[File:]] reference
  70. </li>
  71. <li>
  72. <code>[ 20 MB; 05 m]</code> Run the file_regy command. This will create a table called oimg_lnki_regy which aggregates all info in oimg_lnki_temp. It also creates a table called oimg_file_regy which uses the data generated in the images and redirect commands.
  73. </li>
  74. <li>
  75. <code>[ 05 MB; 01 m]</code> Run the xfer_regy command. This will create a table called oimg_xfer_regy which generates a unique list of all thumbnails to download
  76. </li>
  77. </ul>
  78. <pre>
  79. app.bldr.cmds {
  80. add ('commons.wikimedia.org', 'oimg.image');
  81. add ('commons.wikimedia.org', 'oimg.redirect') {commit_interval = 10000; progress_interval = 100; cleanup_interval = 100;}
  82. add ('simple.wikipedia.org' , 'oimg.image');
  83. add ('simple.wikipedia.org' , 'oimg.redirect') {commit_interval = 10000; progress_interval = 100; cleanup_interval = 100;}
  84. add ('simple.wikipedia.org' , 'oimg.lnki_temp') {commit_interval = 10000; progress_interval = 100; cleanup_interval = 100;}
  85. add ('simple.wikipedia.org' , 'oimg.file_regy');
  86. add ('simple.wikipedia.org' , 'oimg.xfer_regy');
  87. }
  88. app.bldr.run;
  89. </pre>
  90. </div>
  91. </div>
  92. </div>
  93. <div id="mw-head" class="noprint">
  94. <div id="left-navigation">
  95. <div id="p-namespaces" class="vectorTabs">
  96. <h3>Namespaces</h3>
  97. <ul>
  98. <li id="ca-nstab-main" class="selected"><span><a id="ca-nstab-main-href" href="index.html">Page</a></span></li>
  99. </ul>
  100. </div>
  101. </div>
  102. </div>
  103. <div id='mw-panel' class='noprint'>
  104. <div id='p-logo'>
  105. <a style="background-image: url(https://gnosygnu.github.io/xowa/xowa_logo.png);" href="http://xowa.org/" title="Visit the main page"></a>
  106. </div>
  107. <div class="portal" id='xowa-portal-home'>
  108. <h3>XOWA</h3>
  109. <div class="body">
  110. <ul>
  111. <li><a href="http://xowa.org/index.html" title='Visit the main page'>Main page</a></li>
  112. <li><a href="http://xowa.org/screenshots.html" title='See screenshots of XOWA'>Screenshots</a></li>
  113. <li><a href="https://www.youtube.com/watch?v=q0qbXYXEH6M" title="See a video of XOWA Desktop in action">Video</a></li>
  114. <li><a href="http://xowa.org/home/wiki/Help/Download_XOWA.html" title='Download the XOWA application'>Download XOWA</a></li>
  115. <li><a href="http://xowa.org/home/wiki/Dashboard/Image_databases.html" title='Download offline wikis and image databases'>Download wikis</a></li>
  116. </ul>
  117. </div>
  118. </div>
  119. <div class="portal" id='xowa-portal-started'>
  120. <h3>Getting started</h3>
  121. <div class="body">
  122. <ul>
  123. <li><a href="http://xowa.org/home/wiki/App/Setup/System_requirements.html" title='Get XOWA&apos;s system requirements'>Requirements</a></li>
  124. <li><a href="http://xowa.org/home/wiki/App/Setup/Installation.html" title='Get instructions for installing XOWA'>Installation</a></li>
  125. <li><a href="http://xowa.org/home/wiki/App/Import/Simple_Wikipedia.html" title='Learn how to set up Simple Wikipedia'>Simple Wikipedia</a></li>
  126. <li><a href="http://xowa.org/home/wiki/App/Import/English_Wikipedia.html" title='Learn how to set up English Wikipedia'>English Wikipedia</a></li>
  127. <li><a href="http://xowa.org/home/wiki/App/Import/Other_wikis.html" title='Learn how to set up other Wikipedias'>Other Wikipedias</a></li>
  128. </ul>
  129. </div>
  130. </div>
  131. <div class="portal" id='xowa-portal-android'>
  132. <h3>Android</h3>
  133. <div class="body">
  134. <ul>
  135. <li><a href="http://xowa.org/home/wiki/Android/Setup.html" title='Setup XOWA on your Android device'>Setup</a></li>
  136. <li><a href="https://www.youtube.com/watch?v=jsMTBxGweUw" title="See a video of XOWA Android in action">Video</a></li>
  137. </ul>
  138. </div>
  139. </div>
  140. <div class="portal" id='xowa-portal-help'>
  141. <h3>Help</h3>
  142. <div class="body">
  143. <ul>
  144. <li><a href="http://xowa.org/home/wiki/Help/About.html" title='Get more information about XOWA'>About</a></li>
  145. <li><a href="http://xowa.org/home/wiki/Help/Contents.html" title='View a list of help topics'>Contents</a></li>
  146. <li><a href="http://xowa.org/home/wiki/Help/Media.html" title='Read what others have written about XOWA'>Media</a></li>
  147. <li><a href="http://xowa.org/home/wiki/Help/Feedback.html" title='Questions? Comments? Leave feedback for XOWA'>Feedback</a></li>
  148. </ul>
  149. </div>
  150. </div>
  151. <div class="portal" id='xowa-portal-blog'>
  152. <h3>Blog</h3>
  153. <div class="body">
  154. <ul>
  155. <li><a href="http://xowa.org/home/wiki/Blog.html" title='Follow XOWA''s development process'>Current</a></li>
  156. </ul>
  157. </div>
  158. </div>
  159. <div class="portal" id='xowa-portal-links'>
  160. <h3>Links</h3>
  161. <div class="body">
  162. <ul>
  163. <li><a href="http://dumps.wikimedia.org/backup-index.html" title="Get wiki datababase dumps directly from Wikimedia">Wikimedia dumps</a></li>
  164. <li><a href="https://archive.org/search.php?query=xowa" title="Search archive.org for XOWA files">XOWA @ archive.org</a></li>
  165. <li><a href="http://en.wikipedia.org" title="Visit Wikipedia (and compare to XOWA!)">English Wikipedia</a></li>
  166. </ul>
  167. </div>
  168. </div>
  169. <div class="portal" id='xowa-portal-donate'>
  170. <h3>Donate</h3>
  171. <div class="body">
  172. <ul>
  173. <li><a href="https://archive.org/donate/index.php" title="Support archive.org!">archive.org</a></li><!-- listed first due to recent fire damages: http://blog.archive.org/2013/11/06/scanning-center-fire-please-help-rebuild/ -->
  174. <li><a href="https://donate.wikimedia.org/wiki/Special:FundraiserRedirector" title="Support Wikipedia!">Wikipedia</a></li>
  175. <li><a href="http://xowa.org/home/wiki/Help/Donate.html" title="Support XOWA!">XOWA</a></li>
  176. </ul>
  177. </div>
  178. </div>
  179. </div>
  180. </body>
  181. </html>