# Options/Import

From XOWA: the free, open-source, offline wiki application

Apps (basic) | Apps (advanced) | Wiki | HTML | External apps | Modules | Extensions | Misc

- Import
- Files
- Category
- Import Dansguardian

Save

## Wiki setup

Search version: [1] |
Page storage format: [2] |

## Import process

Dump servers: [3] | https://dumps.wikimedia.org/, http://dumps.wikimedia.your.org/, http://wikipedia.c3sl.ufpr.br/, http://ftp.fi.muni.cz/pub/wikimedia/
Import bz2 by stdout: [4] |
Import bz2 by stdout process:[5] |
Custom wiki commands: [6] |
Download xowa_common.css: [7] |
Delete xml file after import: [8] |

## PageRank

PageRank iteration max: [9] |

## Database layout

Max file size for single text database: [10] |

## Decompression apps

Decompress bz2 file [11] |
Decompress zip file [12] |
Decompress gz file [13] |

## Notes

1. ^ Choose one of the following: (default is `2`)
   - 1 : search uses page title data. searches are less accurate, but less space is used
   - 2 : (default) search uses title word data. searches are more accurate, but more space is used and setup is longer. For English Wikipedia, another 1.8 GB and 38 min of time are needed.
2. ^ Choose one of the following: (default is `.gz`)
   - text: fastest for reading but has no compression. Simple Wikipedia will be 300 MB
   - gzip: (default) fast for reading and has compression. Simple Wikipedia will be 100 MB
   - bzip2: very slow for reading but has best compression. Simple Wikipedia will be 85 MB (Note: The performance is very noticeable. Please try this with Simple Wikipedia first before using on a large wiki.)
3. ^ Enter a list of server urls separated by a comma and newline.
   - The default value is:

       http://dumps.wikimedia.your.org/,
       http://dumps.wikimedia.org/,
       http://wikipedia.c3sl.ufpr.br/,
       http://ftp.fi.muni.cz/pub/wikimedia/

   - Note that servers are prioritized from left-to-right. In the default example, your.org will be tried first. If it is offline, then the next server -- dumps.wikimedia.org -- will be tried, etc.
   - See App/Import/Download/Dump_servers for more info
4. ^ NOTE 1: this option only applies if the "Custom wiki commands" option is `wiki.download,wiki.import` (wiki.unzip must be removed)
   Select the method for importing a wiki dump bz2 file. (default is `checked`)
   - checked : import through a native process's stdout. This will be faster, but may not work on all Operating Systems. A 95 MB file takes 85 seconds
   - unchecked: import though Apache Common's Java bz2 compression library. This will be slower, but will work on all Operating Systems. A 95 MB file takes 215 seconds.
   NOTE 2: lbzip2 (Many thanks to Anselm for making this suggestion, as well as compiling the data to support it. See http://sourceforge.net/p/xowa/tickets/263/?limit=10&page=6#f2fb/dcb6) Linux users should consider using lbzip2, as lbzip2 has significant performance differences (30% in many cases).
   - install lbzip2
     - (Debian) `sudo apt-get install lbzip2`
   - change "Import bz2 by stdout process" to
     - `lbzip2`
     - `-dkc "~{src}"`
5. ^ Process used to decompress bz2 by stdout. Recommended: Operating System default
6. ^ Select custom commands: (default is `wiki.download,wiki.unzip,wiki.import`)
   Short version:
   - For fast imports, but high disk space usage, use `wiki.download,wiki.unzip,wiki.import`
   - For slow imports, but low disk space usage, use `wiki.download,wiki.import`
   Long version: Enter a list of commands separated by a comma. Valid commands are listed below. Note that simple.wikipedia.org is used for all examples, but the commands apply to any wiki.
   - `wiki.download`: downloads the wiki data dump from the dump server
     A file will be generated in "/xowa/wiki/simple.wikipedia.org/simplewiki-latest-pages-articles.xml.bz2"
   - `wiki.unzip`: unzips an xml file from the wiki data dump
     A file will be created for "/xowa/wiki/simple.wikipedia.org/simplewiki-latest-pages-articles.xml" (assuming the corresponding .xml.bz2 exists)
     If this step is omitted, then XOWA will read directly from the .bz2 file. Although this will use less space (no .xml file to unzip), it will be significantly slower. Also, due to a program limitation, the progress percentage will not be accurate. It may hover at 99.99% for several minutes
   - `wiki.import`: imports the xml file
     A wiki will be imported from "/xowa/wiki/simple.wikipedia.org/simplewiki-latest-pages-articles.xml"
   The following lists possible combinations:
   - `wiki.download,wiki.unzip,wiki.import` AKA: fastest
     This is the default. Note that this will be the fastest to set up, but will take more space. For example, English Wikipedia will set up in 5 hours and require at least 45 GB of temp space
   - `wiki.download,wiki.import` AKA: smallest
     This will read directly from the bz2 file. Note that this will use the least disk space, but will take more time. For example, English Wikipedia will set up in 8 hours but will only use 5 GB of temp space
7. ^ Affects the xowa_common.css in /xowa/user/anonymous/wiki/wiki_name/html/. Occurs when importing a wiki. (default is `checked`)
   - checked : downloads xowa_common.css from the Wikimedia servers. Note that this stylesheet will be the latest copy but it may cause unexpected formatting in XOWA.
   - unchecked: (default) copies xowa_common.css from /xowa/bin/any/html/html/import/. Note that this stylesheet is the one XOWA is coded against. It is the most stable, but will not have the latest logo
8. ^ (Only relevant for wiki.unzip) Choose one of the following: (default is `checked`)
   - checked : (default) the .xml file is automatically deleted once the import process completes
   - unchecked: the .xml file is untouched
9. ^ Specify one of the following: (default is `0`)
   - 0 : (default) page rank is disabled
   - (number greater than 1): page rank will be calculated until it is finished or maximum number of interations are reached. For more info, see Help/Features/Search/Build
10. ^ Enter a number in MB to represent the cutoff for multi-file wikis and single-file wikis (default is `1500`)
    - For example, 1500 means that a wiki with a dump file size of 1.5 GB or less will generate a single-file wiki (for example, Simple Wikipedia). Any wiki with a dump file size larger than 1.5 GB will generate a multiple-file wiki. (for example, English Wikipedia).
    - If you always want to generate a single-file wiki, set the value to a large number like 999,999 (999 GB)
    - If you always want to generate a multi-file wiki, set the value to 0.
11. ^ Decompress bz2 file(needed for importing dumps) . Recommended: 7-zip
12. ^ Decompress zip file(needed for importing dumps) . Recommended: 7-zip
13. ^ Decompress gz file(needed for importing dumps) . Recommended: 7-zip

### XOWA

- Main page
- Screenshots
- Download XOWA
- Download wikis

### Getting started

- Requirements
- Installation
- Simple Wikipedia
- English Wikipedia
- Other Wikipedias

### Android

- Setup

### Help

- About
- Contents
- Media
- Feedback

### Blog

- Current

### Links

- Wikimedia dumps
- XOWA @ archive.org
- English Wikipedia

### Donate

- archive.org
- Wikipedia