anki/rslib/proto/rust.rs

106 lines
3.5 KiB
Rust
Raw Normal View History

Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
// Copyright: Ankitects Pty Ltd and contributors
// License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
use std::env;
use std::path::Path;
use std::path::PathBuf;
use anki_io::create_dir_all;
use anki_io::read_file;
use anki_io::write_file_if_changed;
use anki_proto_gen::add_must_use_annotations;
use anki_proto_gen::determine_if_message_is_empty;
Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
use anyhow::Context;
use anyhow::Result;
use prost_reflect::DescriptorPool;
pub fn write_rust_protos(descriptors_path: PathBuf) -> Result<DescriptorPool> {
Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
set_protoc_path();
let proto_dir = PathBuf::from("../../proto");
let paths = gather_proto_paths(&proto_dir)?;
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let tmp_descriptors = out_dir.join("descriptors.tmp");
prost_build::Config::new()
.out_dir(&out_dir)
.file_descriptor_set_path(&tmp_descriptors)
.type_attribute(
"Deck.Filtered.SearchTerm.Order",
"#[derive(strum::EnumIter)]",
)
.type_attribute(
"Deck.Normal.DayLimit",
"#[derive(Copy, Eq, serde::Deserialize, serde::Serialize)]",
)
.type_attribute("HelpPageLinkRequest.HelpPage", "#[derive(strum::EnumIter)]")
.type_attribute("CsvMetadata.Delimiter", "#[derive(strum::EnumIter)]")
.type_attribute(
"Preferences.BackupLimits",
"#[derive(Copy, serde::Deserialize, serde::Serialize)]",
)
.type_attribute(
"CsvMetadata.DupeResolution",
"#[derive(serde::Deserialize, serde::Serialize)]",
)
.type_attribute(
"CsvMetadata.MatchScope",
"#[derive(serde::Deserialize, serde::Serialize)]",
)
Merging Notetypes on Import (#2612) * Remember original id when importing notetype * Reuse notetypes with matching original id * Add field and template ids * Enable merging imported notetypes * Fix test Note should be updated if the incoming note's notetype is remapped to the existing note's notetype. On the other hand, it should be skipped if its notetype id is mapped to some new notetype. * Change field and template ids to i32 * Add merge notetypes flag to proto message * Add dialog for apkg import * Move HelpModal into components * Generalize import dialog * Move SettingTitle into components * Add help modal to ImportAnkiPackagePage * Move SwitchRow into components * Fix backend method import * Make testable in browser * Fix broken modal * Wrap in container and fix margins * Update commented Anki version of new proto fields * Check ids when comparing notetype schemas * Add tooltip for merging notetypes. * Allow updating notes regardless of mtime * Gitignore yarn-error.log * Allow updating notetypes regardless of mtime * Fix apkg help carousel * Use i64s for template and field ids * Add option to omit importing scheduling info * Restore last settings in apkg import dialog * Display error when getting metadata in webview * Update manual links for apkg importing * Apply suggestions from code review Co-authored-by: Damien Elmes <dae@users.noreply.github.com> * Omit schduling -> Import all cards as new cards * Tweak importing-update-notes-help * UpdateCondition → ImportAnkiPackageUpdateCondition * Load keyboard.ftl * Skip updating dupes in 'update alwyas' case * Explain more when merging notetypes is required * "omit scheduling" → "with scheduling" * Skip updating notetype dupes if 'update always' * Merge duplicated notetypes from previous imports * Fix rebase aftermath * Fix panic when merging * Clarify 'update notetypes' help * Mention 'merge notetypes' in the log * Add a test which covers the previously panicking path * Use nested ftl messages to ensure consistency * Make order of merged fields deterministic * Rewrite test to trigger panic * Update version comment on new fields
2023-09-09 01:00:55 +02:00
.type_attribute(
"ImportAnkiPackageUpdateCondition",
"#[derive(serde::Deserialize, serde::Serialize)]",
)
Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
.compile_protos(paths.as_slice(), &[proto_dir])
.context("prost build")?;
let descriptors = read_file(&tmp_descriptors)?;
create_dir_all(
descriptors_path
.parent()
.context("missing parent of descriptor")?,
)?;
write_file_if_changed(descriptors_path, &descriptors)?;
Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
let pool = DescriptorPool::decode(descriptors.as_ref())?;
add_must_use_annotations(
&out_dir,
|path| path.file_name().unwrap().starts_with("anki."),
|path, name| determine_if_message_is_empty(&pool, path, name),
)?;
Refactor service generation (#2552) * Automatically elide empty inputs and outputs to backend methods * Refactor service generation Despite the fact that the majority of our Protobuf service methods require an open collection, they were not accessible with just a Collection object. To access the methods (e.g. because we haven't gotten around to exposing the correct API in Collection yet), you had to wrap the collection in a Backend object, and pay a mutex-acquisition cost for each call, even if you have exclusive access to the object. This commit migrates the majority of service methods to the Collection, so they can now be used directly, and improves the ergonomics a bit at the same time. The approach taken: - The service generation now happens in rslib instead of anki_proto, which avoids the need for trait constraints and associated types. - Service methods are assumed to be collection-based by default. Instead of implementing the service on Backend, we now implement it on Collection, which means our methods no longer need to use self.with_col(...). - We automatically generate methods in Backend which use self.with_col() to delegate to the Collection method. - For methods that are only appropriate for the backend, we add a flag in the .proto file. The codegen uses this flag to write the method into a BackendFooService instead of FooService, which the backend implements. - The flag can also allows us to define separate implementations for collection and backend, so we can e.g. skip the collection mutex in the i18n service while also providing the service on a collection.
2023-06-19 07:33:40 +02:00
Ok(pool)
}
fn gather_proto_paths(proto_dir: &Path) -> Result<Vec<PathBuf>> {
let subfolders = &["anki"];
let mut paths = vec![];
for subfolder in subfolders {
for entry in proto_dir.join(subfolder).read_dir().unwrap() {
let entry = entry.unwrap();
let path = entry.path();
if path
.file_name()
.unwrap()
.to_str()
.unwrap()
.ends_with(".proto")
{
println!("cargo:rerun-if-changed={}", path.to_str().unwrap());
paths.push(path);
}
}
}
paths.sort();
Ok(paths)
}
/// Set PROTOC to the custom path provided by PROTOC_BINARY, or add .exe to
/// the standard path if on Windows.
fn set_protoc_path() {
if let Ok(custom_protoc) = env::var("PROTOC_BINARY") {
env::set_var("PROTOC", custom_protoc);
} else if let Ok(bundled_protoc) = env::var("PROTOC") {
if cfg!(windows) && !bundled_protoc.ends_with(".exe") {
env::set_var("PROTOC", format!("{bundled_protoc}.exe"));
}
}
}