Update readme for compilation database support

README.md

@@ -136,8 +136,10 @@ Installation
 
 ### Mac OS X
 
-Please refer to the full Installation Guide below; the following commands are
-provided on a best-effort basis and may not work for you.
+These instructions (using `install.py`) are the quickest way to install
+YouCompleteMe, however they may not work for everyone. If the following
+instructions don't work for you, check out the [full installation
+guide](#full-installation-guide).
 
 Install the latest version of [MacVim][]. Yes, MacVim. And yes, the _latest_.
 
@@ -209,8 +211,10 @@ that are conservatively turned off by default that you may want to turn on.
 
 ### Ubuntu Linux x64
 
-Please refer to the full Installation Guide below; the following commands are
-provided on a best-effort basis and may not work for you.
+These instructions (using `install.py`) are the quickest way to install
+YouCompleteMe, however they may not work for everyone. If the following
+instructions don't work for you, check out the [full installation
+guide](#full-installation-guide).
 
 Make sure you have Vim 7.4.143 with Python 2 or Python 3 support. Ubuntu 14.10
 and later have a Vim that's recent enough. You can see the version of Vim
@@ -271,8 +275,10 @@ that are conservatively turned off by default that you may want to turn on.
 
 ### Fedora Linux x64
 
-Please refer to the full Installation Guide below; the following commands are
-provided on a best-effort basis and may not work for you.
+These instructions (using `install.py`) are the quickest way to install
+YouCompleteMe, however they may not work for everyone. If the following
+instructions don't work for you, check out the [full installation
+guide](#full-installation-guide).
 
 Make sure you have Vim 7.4.143 with Python 2 or Python 3 support. Fedora 21 and
 later have a Vim that's recent enough. You can see the version of Vim installed
@@ -333,8 +339,10 @@ that are conservatively turned off by default that you may want to turn on.
 
 ### Windows
 
-Please refer to the full Installation Guide below; the following commands are
-provided on a best-effort basis and may not work for you.
+These instructions (using `install.py`) are the quickest way to install
+YouCompleteMe, however they may not work for everyone. If the following
+instructions don't work for you, check out the [full installation
+guide](#full-installation-guide).
 
 **Important:** we assume that you are using the `cmd.exe` command prompt and
 that you know how to add an executable to the PATH environment variable.
@@ -412,9 +420,12 @@ that are conservatively turned off by default that you may want to turn on.
 
 ### FreeBSD/OpenBSD
 
-Please refer to the full Installation Guide below; the following commands are
-provided on a best-effort basis and may not work for you. OpenBSD / FreeBSD are
-not officially supported platforms by YCM.
+These instructions (using `install.py`) are the quickest way to install
+YouCompleteMe, however they may not work for everyone. If the following
+instructions don't work for you, check out the [full installation
+guide](#full-installation-guide).
+
+**NOTE:** OpenBSD / FreeBSD are not officially supported platforms by YCM.
 
 Make sure you have Vim 7.4.143 with Python 2 or Python 3 support.
 
@@ -589,7 +600,7 @@ process.
     `-DUSE_SYSTEM_BOOST=ON` to cmake. This may be necessary on some systems
     where the bundled version of boost doesn't compile out of the box.
 
-    NOTE: We **STRONGLY recommend AGAINST use** of the system boost instead
+    **NOTE:** We **STRONGLY recommend AGAINST use** of the system boost instead
     of the bundled version of boost. Random things may break. Save yourself
     the hassle and use the bundled version of boost.
 
@@ -600,7 +611,7 @@ process.
     `lib`, `include` etc. folders right inside that folder). On Windows, you can
     extract the files from the LLVM+Clang installer using [7-zip][7z-download].
 
-    NOTE: This _only_ works with a _downloaded_ LLVM binary package, not a
+    **NOTE:** This _only_ works with a _downloaded_ LLVM binary package, not a
     custom-built LLVM! See docs below for `EXTERNAL_LIBCLANG_PATH` when using a
     custom LLVM build.
 
@@ -622,7 +633,7 @@ process.
     `-DUSE_SYSTEM_LIBCLANG=ON` to cmake _instead of_ the
     `-DPATH_TO_LLVM_ROOT=...` flag.
 
-    NOTE: We **STRONGLY recommend AGAINST use** of the system libclang instead
+    **NOTE:** We **STRONGLY recommend AGAINST use** of the system libclang instead
     of the upstream compiled binaries. Random things may break. Save yourself
     the hassle and use the upstream pre-built libclang.
 
@@ -800,6 +811,71 @@ string.
 
 ### C-family Semantic Completion
 
+In order to perform semantic analysis such as code completion, `GoTo` and
+diagnostics, YouCompleteMe uses `libclang`. This is the library version of the
+clang compiler, sometimes also referred to as llvm. Like any compiler,
+`libclang` requires a set of compile flags in order to parse your code. Simply
+put: If `libclang` can't parse your code, YouCompleteMe can't provide semantic
+analysis.
+
+There are 2 methods which can be used to provide compile flags to `libclang`:
+
+#### Option 1: Use a [compilation database][compdb]
+
+The easiest way to get YCM to compile your code is to use a compilation
+database.  A compilation database is usually generated by your build system
+(e.g. `CMake`) and contains the compiler invocation for each compilation unit in
+your project.
+
+For information on how to generate a compilation database, see the [clang
+documentation][compdb]. In short:
+
+- If using CMake, add `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` when configuring (or
+  add `set( CMAKE_EXPORT_COMPILE_COMMANDS ON )` to `CMakeLists.txt`) and copy or
+  symlink the generated database to the root of your project.
+- If using Ninja, check out the `compdb` tool (`-t compdb`) in its
+  [docs][ninja-compdb].
+- If using GNU make, check out [Bear][].
+- For other build systems, check out
+  [`.ycm_extra_conf.py`](#option-2-provide-the-flags-manually) below.
+
+If no [`.ycm_extra_conf.py`](#option-2-provide-the-flags-manually) is found,
+and no [`ycm_global_ycm_extra_conf`](#the-gycm_global_ycm_extra_conf-option) is
+configured, YouCompleteMe automatically tries to load a compilation database if
+one is found.
+
+YCM looks for a file named `compile_commands.json` in the directory of the
+opened file or in any directory above it in the hierarchy (recursively); when
+the file is found, it is loaded.  YouCompleteMe performs the following lookups
+when extracting flags for a particular file:
+
+- If the database contains an entry for the file, the flags for that file are
+  used.
+- If the file is a header file and a source file with the same root exists in
+  the database, the flags for the source file are used. For example, if the file
+  is `/home/Test/project/src/lib/something.h` and the database contains an entry
+  for `/home/Test/project/src/lib/something.cc`, then the flags for
+  `/home/Test/project/src/lib/something.cc` are used.
+- Otherwise, if any flags have been returned from the directory containing the
+  requested file, those flags are used. This heuristic is intended to provide
+  potentially working flags for newly created files.
+
+Finally, YCM converts any relative paths in the extracted flags to absolute
+paths. This ensures that compilation can be performed from any Vim working
+directory.
+
+#### Option 2: Provide the flags manually
+
+If you don't have a compilation database, or aren't able to generate one,
+you have to tell YouCompleteMe how to compile your code some other way.
+
+Every c-family project is different. It is not possible for YCM to guess what
+compiler flags to supply for your project. Fortunately, YCM provides a mechanism
+for you to generate the flags for a particular file with _arbitrary complexity_.
+This is achieved by requiring you to provide a Python module which implements a
+trival function which, given the file name as argument, returns a list of
+compiler flags to use to compile that file.
+
 YCM looks for a `.ycm_extra_conf.py` file in the directory of the opened file or
 in any directory above it in the hierarchy (recursively); when the file is
 found, it is loaded (only once!) as a Python module. YCM calls a `FlagsForFile`
@@ -814,20 +890,41 @@ This system was designed this way so that the user can perform any arbitrary
 sequence of operations to produce a list of compilation flags YCM should hand
 to Clang.
 
-[See YCM's own `.ycm_extra_conf.py`][flags_example] for details on how this
-works. You should be able to use it _as a starting point_. **Don't** just
-copy/paste that file somewhere and expect things to magically work; **your project
-needs different flags**. Hint: just replace the strings in the `flags` variable
-with compilation flags necessary for your project. That should be enough for 99%
-of projects.
+**NOTE**: It is highly recommended to include `-x <language>` flag to libclang.
+This is so that the correct language is detected, particularly for header files.
+Common values are `-x c` for C, `-x c++` for C++ and `-x objc` for Objective-C.
+
+To give you an impression, if your c++ project is trivial, and your usual
+compilation command is: `g++ -Wall -Wextra -Werror -o FILE.o FILE.cc`, then the
+following `.ycm_extra_conf.py` is enough to get semantic analysis from
+YouCompleteMe:
+
+```python
+def FlagsForFile( filename, **kwargs ):
+  return {
+    'flags': [ '-x', 'c++', '-Wall', '-Wextra', '-Werror' ],
+  }
+```
+
+As you can see from the trivial example, YCM calls the `FlagsForFile` method,
+passing it the file name. The `**kwargs` is for advanced users and can usually
+be ignored. The `FlagsForFile` function returns a dictionary with a single
+element `'flags'`. This element is a `list` of compiler flags to pass to
+libclang for the file `filename`. That's it! This is actually enough for most
+projects, but for complex projects it is not uncommon to integrate directly with
+an existing build system using the full power of the Python language.
+
+For a more elaborate example,
+[see YCM's own `.ycm_extra_conf.py`][flags_example]. You should be able to use
+it _as a starting point_. **Don't** just copy/paste that file somewhere and
+expect things to magically work; **your project needs different flags**. Hint:
+just replace the strings in the `flags` variable with compilation flags
+necessary for your project. That should be enough for 99% of projects.
 
-Yes, [Clang's `CompilationDatabase` system][compdb] is also supported. Again,
-see the above linked example file. You can get CMake to generate this file for
-you by adding `set( CMAKE_EXPORT_COMPILE_COMMANDS 1 )` to your project's
-`CMakeLists.txt` file (if using CMake). If you're not using CMake, you could use
-something like [Bear][] to generate the `compile_commands.json` file.
+You could also consider using [YCM-Generator][ygen] to generate the
+`ycm_extra_conf.py` file.
 
-Consider using [YCM-Generator][ygen] to generate the `ycm_extra_conf.py` file.
+#### Errors during compilaton
 
 If Clang encounters errors when compiling the header files that your file
 includes, then it's probably going to take a long time to get completions.  When
@@ -1130,8 +1227,8 @@ be fixed by a call to `:YcmCompleter FixIt`, then ` (FixIt available)` is
 appended to the error or warning text. See the `FixIt` completer subcommand for
 more information.
 
-NOTE: The absense of ` (FixIt available)` does not strictly imply a fix-it is
-not available as not all completers are able to provide this indication. For
+**NOTE:** The absense of ` (FixIt available)` does not strictly imply a fix-it
+is not available as not all completers are able to provide this indication. For
 example, the c-sharp completer provides many fix-its but does not add this
 additional indication.
 
@@ -1176,7 +1273,8 @@ section for more information on the available subcommands and their usage.
 YcmCompleter Subcommands
 ------------------------
 
-NOTE: See the docs for the `YcmCompleter` command before tackling this section.
+**NOTE:** See the docs for the `YcmCompleter` command before tackling this
+section.
 
 The invoked subcommand is automatically routed to the currently active semantic
 completer, so `:YcmCompleter GoToDefinition` will invoke the `GoToDefinition`
@@ -1215,10 +1313,10 @@ Supported in filetypes: `c, cpp, objc, objcpp, cs, go, python, rust`
 
 Looks up the symbol under the cursor and jumps to its definition.
 
-NOTE: For C-family languages **this only works in certain situations**, namely when
-the definition of the symbol is in the current translation unit. A translation
-unit consists of the file you are editing and all the files you are including
-with `#include` directives (directly or indirectly) in that file.
+**NOTE:** For C-family languages **this only works in certain situations**,
+namely when the definition of the symbol is in the current translation unit. A
+translation unit consists of the file you are editing and all the files you are
+including with `#include` directives (directly or indirectly) in that file.
 
 Supported in filetypes: `c, cpp, objc, objcpp, cs, go, javascript, python,
 rust, typescript`
@@ -1289,7 +1387,7 @@ For example:
 
 Invoking this command on `s` returns `std::string => std::basic_string<char>`
 
-NOTE: Due to limitations of `libclang`, invoking this command on the word
+**NOTE:** Due to limitations of `libclang`, invoking this command on the word
 `auto` typically returns `auto`. However, invoking it on a usage of the variable
 with inferred type returns the correct type, but typically it is repeated due to
 `libclang` returning that the types differ.
@@ -1303,7 +1401,7 @@ auto x = &s; // invoking on x or auto returns "auto";
 std::cout << *x; // invoking on x returns "const char ** => const char **"
 ```
 
-NOTE: Causes re-parsing of the current translation unit.
+**NOTE:** Causes re-parsing of the current translation unit.
 
 Supported in filetypes: `c, cpp, objc, objcpp, javascript, typescript`
 
@@ -1347,7 +1445,7 @@ context of the second `C::f` is the translation unit.
 
 For global declarations, the semantic parent is the translation unit.
 
-NOTE: Causes re-parsing of the current translation unit.
+**NOTE:** Causes re-parsing of the current translation unit.
 
 Supported in filetypes: `c, cpp, objc, objcpp`
 
@@ -1408,15 +1506,16 @@ also appended to the diagnostic text in the output of the `:YcmDiags` command
 for any diagnostics with available fix-its (where the completer can provide this
 indication).
 
-NOTE: Causes re-parsing of the current translation unit.
+**NOTE:** Causes re-parsing of the current translation unit.
 
-NOTE: After applying a fix-it, the diagnostics UI is not immediately updated.
-This is due to a technical restriction in Vim. Moving the cursor, or issuing
-the `:YcmForceCompileAndDiagnostics` command will refresh the diagnostics.
-Repeated invocations of the `FixIt` command on a given line, however, _do_ apply
-all diagnostics as expected without requiring refreshing of the diagnostics UI.
-This is particularly useful where there are multiple diagnostics on one line, or
-where after fixing one diagnostic, another fix-it is available.
+**NOTE:** After applying a fix-it, the diagnostics UI is not immediately
+updated.  This is due to a technical restriction in Vim. Moving the cursor, or
+issuing the `:YcmForceCompileAndDiagnostics` command will refresh the
+diagnostics.  Repeated invocations of the `FixIt` command on a given line,
+however, _do_ apply all diagnostics as expected without requiring refreshing of
+the diagnostics UI.  This is particularly useful where there are multiple
+diagnostics on one line, or where after fixing one diagnostic, another fix-it is
+available.
 
 Supported in filetypes: `c, cpp, objc, objcpp, cs`
 
@@ -1440,7 +1539,7 @@ When a Refactor or FixIt command touches multiple files, YouCompleteMe attempts
 to apply those modifications to any existing open, visible buffer in the current
 tab. If no such buffer can be found, YouCompleteMe opens the file in a new
 small horizontal split at the top of the current window, applies the change,
-and then *hides* the window. NOTE: The buffer remains open, and must be
+and then *hides* the window. **NOTE:** The buffer remains open, and must be
 manually saved. A confirmation dialog is opened prior to doing this to remind
 you that this is about to happen.
 
@@ -1457,10 +1556,10 @@ can be undone using Vim's powerful undo features (see `:help undo`). Note
 that Vim's undo is per-buffer, so to undo all changes, the undo commands must
 be applied in each modified buffer separately.
 
-NOTE: While applying modifications, Vim may find files which are already open
-and have a swap file. The command is aborted if you select Abort or Quit in any
-such prompts. This leaves the Refactor operation partially complete and must be
-manually corrected using Vim's undo features. The quickfix list is *not*
+**NOTE:** While applying modifications, Vim may find files which are already
+open and have a swap file. The command is aborted if you select Abort or Quit in
+any such prompts. This leaves the Refactor operation partially complete and must
+be manually corrected using Vim's undo features. The quickfix list is *not*
 populated in this case. Inspect `:buffers` or equivalent (see `:help buffers`)
 to see the buffers that were opened by the command.
 
@@ -1592,8 +1691,8 @@ popup menu.
 
 A special value of `0` means there is no limit.
 
-NOTE: This option only applies to the identifier completer; it has no effect on
-the various semantic completers.
+**NOTE:** This option only applies to the identifier completer; it has no effect
+on the various semantic completers.
 
 Default: `0`
 
@@ -1821,7 +1920,7 @@ when the regex (treated as case-insensitive) is found in the diagnostic text.
 - "level": Accepts a string level, either "warning" or "error." This type 
 matches when the diagnostic has the same level.
 
-NOTE: The regex syntax is **NOT** Vim's, it's [Python's][python-re].
+**NOTE:** The regex syntax is **NOT** Vim's, it's [Python's][python-re].
 
 Default: `{}`
 
@@ -2009,9 +2108,9 @@ YCM will by default search for an appropriate Python interpreter on your system.
 You can use this option to override that behavior and force the use of a
 specific interpreter of your choosing.
 
-NOTE: This interpreter is only used for the [ycmd server][ycmd]. The YCM client
-running inside Vim always uses the Python interpreter that's embedded inside
-Vim.
+**NOTE:** This interpreter is only used for the [ycmd server][ycmd]. The YCM
+client running inside Vim always uses the Python interpreter that's embedded
+inside Vim.
 
 Default: `''`
 
@@ -2289,9 +2388,9 @@ let g:ycm_extra_conf_globlist = ['~/dev/*','!~/*']
 * As the first rule takes precedence everything in the home directory excluding
   the `~/dev` directory will be blacklisted.
 
-NOTE: The glob pattern is first expanded with Python's `os.path.expanduser()`
-and then resolved with `os.path.abspath()` before being matched against the
-filename.
+**NOTE:** The glob pattern is first expanded with Python's
+`os.path.expanduser()` and then resolved with `os.path.abspath()` before being
+matched against the filename.
 
 Default: `[]`
 
@@ -2333,7 +2432,7 @@ It's also possible to use a regular expression as a trigger. You have to prefix
 your trigger with `re!` to signify it's a regex trigger. For instance,
 `re!\w+\.` would only trigger after the `\w+\.` regex matches.
 
-NOTE: The regex syntax is **NOT** Vim's, it's [Python's][python-re].
+**NOTE:** The regex syntax is **NOT** Vim's, it's [Python's][python-re].
 
 Default: `[see next line]`
 
@@ -2418,7 +2517,7 @@ Default: `''`
 let g:ycm_python_binary_path = 'python'
 ```
 
-NOTE: the settings above will make YCM use the first `python` executable
+**NOTE:** the settings above will make YCM use the first `python` executable
 found through the PATH.
 
 FAQ
@@ -2633,7 +2732,7 @@ The details aren't important.
 
 The solution is that the version of Python linked and run against must be built
 with either `--enable-shared` or `--enable-framework` (on OS X).
-This is achieved as follows (NOTE: for Mac, replace `--enable-shared`
+This is achieved as follows (**NOTE:** for Mac, replace `--enable-shared`
 with `--enable-framework`):
 
 - When building python from source: `./configure --enable-shared {options}`
@@ -2658,11 +2757,11 @@ output of `ctags --version` should list "Exuberant Ctags".
 Ctags needs to be called with the `--fields=+l` (that's a lowercase `L`, not a
 one) option because YCM needs the `language:<lang>` field in the tags output.
 
-NOTE: [Exuberant Ctags][exuberant-ctags] by default sets language tag for `*.h`
-files as `C++`. If you have C (not C++) project, consider
-giving parameter `--langmap=c:.c.h` to ctags to see tags from `*.h` files.
+**NOTE:** [Exuberant Ctags][exuberant-ctags] by default sets language tag for
+`*.h` files as `C++`. If you have C (not C++) project, consider giving parameter
+`--langmap=c:.c.h` to ctags to see tags from `*.h` files.
 
-NOTE: Mac OS X comes with "plain" ctags installed by default. `brew install
+**NOTE:** Mac OS X comes with "plain" ctags installed by default. `brew install
 ctags` will get you the Exuberant Ctags version.
 
 Also make sure that your Vim `tags` option is set correctly. See `:h 'tags'` for
@@ -2946,7 +3045,7 @@ This software is licensed under the [GPL v3 license][gpl].
 [issue-669]: https://github.com/Valloric/YouCompleteMe/issues/669
 [status-mes]: https://groups.google.com/forum/#!topic/vim_dev/WeBBjkXE8H8
 [python-re]: https://docs.python.org/2/library/re.html#regular-expression-syntax
-[bear]: https://github.com/rizsotto/Bear
+[Bear]: https://github.com/rizsotto/Bear
 [Options]: https://github.com/Valloric/YouCompleteMe#options
 [ygen]: https://github.com/rdnetto/YCM-Generator
 [Gocode]: https://github.com/nsf/gocode
@@ -2975,3 +3074,4 @@ This software is licensed under the [GPL v3 license][gpl].
 [vim_win-python2.7.11-bug]: https://github.com/vim/vim/issues/717
 [vim_win-python2.7.11-bug_workaround]: https://github.com/vim/vim-win32-installer/blob/master/appveyor.bat#L90
 [gitter]: https://gitter.im/Valloric/YouCompleteMe
+[ninja-compdb]: https://ninja-build.org/manual.html