A couple of motivations for this:
- genbackend.py was somewhat messy, and difficult to change with the
lack of types. The mobile clients used it as a base for their generation,
so improving it will make life easier for them too, once they're ported.
- It will make it easier to write a .ts generator in the future
- We currently implement a bunch of helper methods on protobuf types
which don't allow us to compile the protobuf types until we compile
the Anki crate. If we change this in the future, we will be able to
do more of the compilation up-front.
We no longer need to record the services in the proto file, as we can
extract the service order from the compiled protos. Support for map types
has also been added.
Running and testing should be working on the three platforms, but
there's still a fair bit that needs to be done:
- Wheel building + testing in a venv still needs to be implemented.
- Python requirements still need to be compiled with piptool and pinned;
need to compile on all platforms then merge
- Cargo deps in cargo/ and rslib/ need to be cleaned up, and ideally
unified into one place
- Currently using rustls to work around openssl compilation issues
on Linux, but this will break corporate proxies with custom SSL
authorities; need to conditionally use openssl or use
https://github.com/seanmonstar/reqwest/pull/1058
- Makefiles and docs still need cleaning up
- It may make sense to reparent ts/* to the top level, as we don't
nest the other modules under a specific language.
- rspy and pylib must always be updated in lock-step, so merging
rspy into pylib as a private module would simplify things.
- Merging desktop-ftl and mobile-ftl into the core ftl would make
managing and updating translations easier.
- Obsolete scripts need removing.
- And probably more.
The previous implementation had some slightly questionable memory safety
properties (older versions of PyO3 didn't uphold the Rust aliasing rules
and would thus create multiple &mut references to #[pyclass] objects).
This explains why Backend has internal Mutex<T>s even though all of its
methods took &mut self.
The solution is to simply make all methods take &self, which luckily
doesn't pose too make issues -- most of the code inside Backend already
has sufficient locking. The only two things which needed to be
explicitly handled where:
1. "self.runtime" which was fairly easy to handle. All usages of
the Runtime only require an immutable reference to create a new
Handle, so we could switch to OnceCell which provides
lazy-initialisation semantics without needing a more heavy-handed
Mutex<tokio::runtime::Handle>.
2. "self.sync_abort" was simply wrapped in a Mutex<>, though some of the
odd semantics of sync_abort (not being able to handle multiple
processes synchronising at the same time) become pretty obvious with
this change (for now we just log a warning in that case). In
addition, switch to an RAII-style guard to make sure we don't forget
to clear the abort_handle.
As a result, we now no longer break Rust's aliasing rules and we can
build with newer versions of PyO3 which have runtime checks for these
things (and build on stable Rust).
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
prost-build doesn't generate rustfmt-safe code, so we had to add it to
the ignore list for rustfmt on "make check". However, the ignore list
isn't supported by stable rustfmt, so we have to work around this some
other way -- in this case, just do "rustfmt" on the generated file in
"build.rs" (this way, formatting errors in checked-in source code are
still caught but generated code doesn't cause spurrious errors).
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
