2021-01-31 21:50:21 +01:00
|
|
|
// Copyright: Ankitects Pty Ltd and contributors
|
|
|
|
|
// License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
|
|
|
|
|
|
2021-07-10 12:44:22 +02:00
|
|
|
macro_rules! protobuf {
|
|
|
|
|
($ident:ident) => {
|
|
|
|
|
pub mod $ident {
|
2022-09-24 03:12:58 +02:00
|
|
|
#![allow(clippy::derive_partial_eq_without_eq)]
|
2021-07-10 12:44:22 +02:00
|
|
|
include!(concat!(
|
|
|
|
|
env!("OUT_DIR"),
|
|
|
|
|
concat!("/anki.", stringify!($ident), ".rs")
|
|
|
|
|
));
|
|
|
|
|
}
|
|
|
|
|
pub use $ident::*;
|
|
|
|
|
};
|
2021-07-10 11:52:31 +02:00
|
|
|
}
|
refactor protobuf handling for split/import
In order to split backend.proto into a more manageable size, the protobuf
handling needed to be updated. This took more time than I would have
liked, as each language handles protobuf differently:
- The Python Protobuf code ignores "package" directives, and relies
solely on how the files are laid out on disk. While it would have been
nice to keep the generated files in a private subpackage, Protobuf gets
confused if the files are located in a location that does not match
their original .proto layout, so the old approach of storing them in
_backend/ will not work. They now clutter up pylib/anki instead. I'm
rather annoyed by that, but alternatives seem to be having to add an extra
level to the Protobuf path, making the other languages suffer, or trying
to hack around the issue by munging sys.modules.
- Protobufjs fails to expose packages if they don't start with a capital
letter, despite the fact that lowercase packages are the norm in most
languages :-( This required a patch to fix.
- Rust was the easiest, as Prost is relatively straightforward compared
to Google's tools.
The Protobuf files are now stored in /proto/anki, with a separate package
for each file. I've split backend.proto into a few files as a test, but
the majority of that work is still to come.
The Python Protobuf building is a bit of a hack at the moment, hard-coding
"proto" as the top level folder, but it seems to get the job done for now.
Also changed the workspace name, as there seems to be a number of Bazel
repos moving away from the more awkward reverse DNS naming style.
2021-07-10 09:50:18 +02:00
|
|
|
|
2021-07-10 12:44:22 +02:00
|
|
|
protobuf!(backend);
|
2021-07-10 13:58:34 +02:00
|
|
|
protobuf!(card_rendering);
|
2021-07-10 12:44:22 +02:00
|
|
|
protobuf!(cards);
|
|
|
|
|
protobuf!(collection);
|
2021-07-11 11:35:18 +02:00
|
|
|
protobuf!(config);
|
2021-07-10 13:33:12 +02:00
|
|
|
protobuf!(deckconfig);
|
|
|
|
|
protobuf!(decks);
|
|
|
|
|
protobuf!(generic);
|
|
|
|
|
protobuf!(i18n);
|
Colpkg fixes (#1722)
* Fix legacy colpkg import; disable v3 import/export; add roundtrip test
The test has revealed we weren't decompressing the media files on v3
import. That's easy to fix, but means all files need decompressing
even when they already exist, which is not ideal - it would be better
to store size/checksum in the metadata instead.
* Switch media and meta to protobuf; re-enable v3 import/export
- Fixed media not being decompressed on import
- The uncompressed size and checksum is now included for each media
entry, so that we can quickly check if a given file needs to be extracted.
We're still just doing a naive size comparison on colpkg import at the
moment, but we may want to use a checksum in the future, and will need
a checksum for apkg imports.
- Checksums can't be efficiently encoded in JSON, so the media list
has been switched to protobuf to reduce the the space requirements.
- The meta file has been switched to protobuf as well, for consistency.
This will mean any colpkg files exported with beta7 will be
unreadable.
* Avoid integer version comparisons
* Re-enable v3 test
* Apply suggestions from code review
Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
* Add export_colpkg() method to Collection
More discoverable, and easier to call from unit tests
* Split import/export code out into separate folders
Currently colpkg/*.rs contain some routines that will be useful for
apkg import/export as well; in the future we can refactor them into a
separate file in the parent module.
* Return a proper error when media import fails
This tripped me up when writing the earlier unit test - I had called
the equivalent of import_colpkg()?, and it was returning a string error
that I didn't notice. In practice this should result in the same text
being shown in the UI, but just skips the tooltip.
* Automatically create media folder on import
* Move roundtrip test into separate file; check collection too
* Remove zstd version suffix
Prevents a warning shown each time Rust Analyzer is used to check the
code.
Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
2022-03-17 06:11:23 +01:00
|
|
|
protobuf!(import_export);
|
2021-07-22 10:05:38 +02:00
|
|
|
protobuf!(links);
|
2021-07-10 13:58:34 +02:00
|
|
|
protobuf!(media);
|
2021-07-10 13:33:12 +02:00
|
|
|
protobuf!(notes);
|
|
|
|
|
protobuf!(notetypes);
|
|
|
|
|
protobuf!(scheduler);
|
|
|
|
|
protobuf!(search);
|
2021-07-10 13:58:34 +02:00
|
|
|
protobuf!(stats);
|
2021-07-10 13:33:12 +02:00
|
|
|
protobuf!(sync);
|
2021-07-10 13:58:34 +02:00
|
|
|
protobuf!(tags);
|