anki/README.development

Anki development
=================

Packaged betas
---------------

For non-developers who want to try this development code, the easiest way is
to use a packaged version - please see:

https://betas.ankiweb.net/#/

You are welcome to run Anki from source instead, but it is expected that you
can sort out all dependencies and issues by yourself - we are not able to
provide support for problems you encounter when running from source.

Python wheels
--------------

If you want to run Anki from a local Python installation but don't want
to make changes to the source code, you can install pre-built packages from PyPI.

$ python -m venv pyenv
$ pyenv/bin/pip install aqt anki ankirspy pyqt5 pyqtwebengine
$ pyenv/bin/python -c 'import aqt; aqt.run()'

Building from source
--------------------

You will need the following dependencies. Some OS-specific tips on installing
them are further down this document.

 - Python 3.7+
 - portaudio
 - mpv
 - lame
 - npm
 - your platform's C compiler, eg gcc, Xcode or Visual Studio 2017.
 - GNU make
 - protoc v3 (https://github.com/protocolbuffers/protobuf/releases)
 - rustup (https://rustup.rs/)
 - gettext
 - rsync
 - perl
 - ripgrep (cargo install ripgrep)
 - git
 - curl

Once you've installed the above components, execute ./run in this repo,
which will build the subcomponents, and start Anki. Any arguments included
on the command line will be passed on to Anki. The first run will take
quite a while to download and build everything - please be patient.

Don't name the Git checkout ~/Anki or ~/Documents/Anki, as those folders
were used on old Anki versions and will be automatically moved.

Before contributing code, please read README.contributing.

If you'd like to contribute translations, please see the translations section
of http://ankisrs.net/docs/manual.html#_contributing

Subcomponents
--------------

- pylib contains a Python module (anki) with the non-GUI Python code.
- qt contains the Qt GUI implementation (aqt).
- rspy contains a Python module (ankirspy) for accessing the Rust code.
- rslib contains the parts of the code implemented in Rust.
- proto contains the interface used to communicate between different
  languages.

The pyenv folder is created when running make for the first time.
It is a Python virtual environment that contains Anki's libraries
and all the required dependencies.

Makefile
--------------

Use 'make check' to run unit tests, type checking and linting on all of the
subcomponents.

Use 'make fix' to fix any formatting issues that were found with 'make check'.

Use 'make build' to output Python wheels of the subcomponents into the dist/
folder.

Use 'make clean' to remove some generated files.

To see all commands run by make or any shell script, export the environment
variable SHELLFLAGS with '-x' to tell shell to print all commands run by it.
For example, 'export SHELLFLAGS=-x' on Linux or 'set SHELLFLAGS=-x' on Windows.

PyQt
-----

The build scripts will use PyQt/Qt from PyPI by default. If you wish to use a
system install, you will need to set up the pyenv folder yourself, making sure
you have PyQt5, the WebEngine module and development tools (eg pyqt5-dev-tools)
installed as well. You'll need to create the venv with --system-site-packages.

Studying Anki on your Dev machine?
----------------------------------
It's advisable to create a new Anki dev profile for debug and testing so that your study collection doesn't become
incompatible with your installed version:
    - run Anki
    - File | Switch profile > Add > provide [dev profile name] > Ok
    - make a note of that [dev profile name]

Because you now have at least two profiles the Profiles dialog will open when Anki runs to allow you to choose the
profile to load.

Alternatively you can set your Anki project to load your [dev profile name] profile automatically with the startup
options described here: https://docs.ankiweb.net/#/files?id=startup-options

To do this in PyCharm:
    - right click on the "run" file in the root of the PyCharm Anki folder
    - click "Edit 'run'..."
    - in Script options enter:
        - "-p [dev profile name]"
            - without the quotes
        - click "Ok"

Debian/Ubuntu users
-------------------

Install Python 3.7+ if it's not installed.

Install other dependencies:
```
sudo apt install portaudio19-dev mpv lame npm rsync gcc gettext git curl python3-dev python3-venv libxcb-xinerama0
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip
sudo unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local/ -x readme.txt
rustup update
cargo install ripgrep
```

Another way to build Anki on Linux is to use Docker - please see README.docker
for more information.

Mac users
----------

You can use homebrew to install some dependencies:

$ brew install python mpv lame portaudio protobuf npm rustup-init gettext rename ripgrep

$ brew link gettext --force

Windows users (using Visual Studio)
----------

The build scripts assume a UNIX-like environment, so on Windows you will
need to use WSL or Cygwin to use them.

User-contributed instructions for building using Cygwin:

1. Download and install Cygwin and put its `/bin/` directory on your system path (This PC > Properties > Advancded system settings > Environment Variables > double-click Path > New).
1. Install the Cygwin Packages: `apt-cyg install rsync make` OR select rsync package during Cygwin installation
    1. Download `gettext` 0.20.1 or superior and put its `bin` directory on your system path.
        1. https://mlocati.github.io/articles/gettext-iconv-windows.html
1. Download and install Python for Windows (not from Cygwin) and put `python.exe` (not `python3.exe`) on your system path.
1. Download and install pip for your Windows Python (`python -m ensurepip`).
1. Download and install rust (compiler), npm, git and put them your system path.
1. Download and install the pyaudio wheel from: https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio
    1. After download the file for your Python version, you need to define the following environment
        variable before running Anki:
        `set "ANKI_EXTRA_PIP=python -m pip install full/path/to/PyAudio‑0.2.11‑cp38‑cp38‑win_amd64.whl"`
    1. If there is not an wheel available for your Python version, you can built it from the source
        following the installation instructions on: https://github.com/evandroforks/pyaudio
        After building and installing portaudio, you need to define the following environment
        variable before running Anki:
        `set "ANKI_EXTRA_PIP=python -m pip install git+https://github.com/evandroforks/pyaudio"`
1. Open a `cmd.exe` (command prompt) on the Anki repository and run the command `sh run`
    1. Do not use `bash run` because it my call for Windows Subsystem for Linux
    1. Do not use any Cygwin terminal as `mintty.exe` because the `rust lang` compiler does not work with them

Windows users (using PyCharm)
-----------------------------
These instructions assume Microsoft Python support is not installed; neither via VS nor VS Code nor some other way.
    If MS Python support is already installed the process is different, and these notes may not work seamlessly for you.

These notes are proscriptive, other methods will work (such as a different Bash/Shell).

While some of this list is in an arbitrary order, significant steps are in a specific order, therefore it is strongly
    recommended to follow these instructions step-by-step.

- Python and PyCharm are a given if you are choosing this path and should be the first and second step respectively.

- close PyCharm, if it's open.

- install Node.js (npm installs with nodejs)

- install Cygwin (check the Cygwin /bin folder is added to System PATH)

- use the Visual Studio Community installer to install:
	- the individual C++ Clang Compiler for Windows (10.0.0) component.
	- the Python development Workflow

- install rustup (https://rustup.rs/)

- download and install Strawberry Perl msi from http://strawberryperl.com/
    - this is the installer at the time of writing these notes:
    	http://strawberryperl.com/download/5.32.0.1/strawberry-perl-5.32.0.1-64bit.msi

- mpv isn't required unless you are developing against mpv (Anki will revert to Mplayer)
	- open an elevated Git Bash (elevated to "Run as administrator")
	- choco install mpv
		- assumes you have Chocolatey installed

- install Git and Git Bash (this delivers curl too)
    - add git/usr/bin to your System PATH before cygwin/bin folder, mentioned above.
	- run:
		- git config --global core.autocrlf
		- make a note of the result (referred to as [yourLineEndingsValue] later in these notes)
	- if the [yourLineEndingsValue] is not 'input', run (to preserve Anki's Unix line endings):
		- git config --global core.autocrlf input
	- run:
		- cargo install ripgrep

- got to https://github.com/protocolbuffers/protobuf/releases
    - download latest protoc-v.v.v-win64.zip where 'v' is replaced with version numbers
	- this is the file at the time of writing these notes:
		https://github.com/protocolbuffers/protobuf/releases/download/v3.13.0/protoc-3.13.0-win64.zip
	- extract bin\protoc.exe to C:\Program Files\Git\usr\bin (or where ever you installed Git and Git Bash above)

- go to http://repo.msys2.org/msys/x86_64/
	- download latest rsync-v.v.v-v-x86_64.pkg.tar.xz  where 'v' is replaced with version numbers
	- this is the file at the time of writing these notes:
		http://repo.msys2.org/msys/x86_64/rsync-3.1.3-1-x86_64.pkg.tar.xz
	- extract the tar, then extract usr\bin\rsync.exe to c:\Program Files\Git\usr\bin as for protoc.exe above

- install remaining dependencies to Python packages
    - pip install https://github.com/ankitects/windows-ci-tools/PyAudio-0.2.11-cp37-cp37-win_amd64.whl
	- pip install lameenc
	- pip install python-gettext

- clone your GitHub Anki fork to a local folder

- open that local Anki source folder in PyCharm

- default PyCharm Terminal to Git Bash:
	- these steps aren't required, you can just use Git Bash externally to run Anki and the commands below, if you
	    prefer
    - open Anki project in PyCharm
	- open File > Settings
	- navigate to Tools > Terminal
	- set Shell path to: "C:\Program Files\Git\bin\bash.exe" (or where ever you installed Git and Git Bash)
	- restart PyCharm with Anki project

- open PyCharm Terminal (Git Bash)

- set your local Anki project to preserve unix line endings:
	- git config --local core.autocrlf input
- if [yourLineEndingsValue] was not 'input', revert global line endings:
    - git config --global core.autocrlf [yourLineEndingsValue]
		- e.g. (most likely):
			- git config --global core.autocrlf true

- ensure your virtual environment is set up as you like it to be
    - if you don't set the virtual environment to pyenv (Anki's virtual environment folder) don't forget to add the
        virtual environment folder to your global (not local) .gitignore if you haven't already.

- add .idea folder to your global .gitignore (not the local .gitignore in the cloned repo)

- from project root run:
    - pip install -r qt/requirements.qt

- in PyCharm
    - either:
        - open Terminal (Git Bash)
            - type ./run
        - or:
            - right click 'run' file in Anki project root and click the command to run it
    - watch the magic happen (it takes a while and there are a couple of long pauses).

please note: these instructions have been based off the general notes above and the notes for installing for Windows
    using Visual Studio. Without those notes this section would have been very difficult, if not impossible.

Environmental Variables
-----------------------

If ANKIDEV is set before starting Anki, some extra log messages will be printed on stdout,
and automatic backups will be disabled - so please don't use this except on a test profile.

If LOGTERM is set before starting Anki, warnings and error messages that are normally placed
in the collection2.log file will also be printed on stdout.