Commit Graph

90 Commits

Author SHA1 Message Date
RumovZ
31b7464c67
Add card meta for persisting custom scheduling state (#2040)
* Add card meta for persisting custom scheduling state

* Rename meta -> custom_data

* Enforce limits on size of custom data

Large values will slow down table scans of the cards table, and it's
easier to be strict now and possibly relax things in the future than
the opposite.

* Pack card states and customData into a single message

+ default customData to empty if it can't be parsed

Co-authored-by: Damien Elmes <gpg@ankiweb.net>
2022-09-02 11:22:49 +10:00
RumovZ
42cbe42f06
Plaintext import/export (#1850)
* Add crate csv

* Add start of csv importing on backend

* Add Menomosyne serializer

* Add csv and json importing on backend

* Add plaintext importing on frontend

* Add csv metadata extraction on backend

* Add csv importing with GUI

* Fix missing dfa file in build

Added compile_data_attr, then re-ran cargo/update.py.

* Don't use doubly buffered reader in csv

* Escape HTML entities if CSV is not HTML

Also use name 'is_html' consistently.

* Use decimal number as foreign ease (like '2.5')

* ForeignCard.ivl → ForeignCard.interval

* Only allow fixed set of CSV delimiters

* Map timestamp of ForeignCard to native due time

* Don't trim CSV records

* Document use of empty strings for defaults

* Avoid creating CardGenContexts for every note

This requires CardGenContext to be generic, so it works both with an
owned and borrowed notetype.

* Show all accepted file types  in import file picker

* Add import_json_file()

* factor → ease_factor

* delimter_from_value → delimiter_from_value

* Map columns to fields, not the other way around

* Fallback to current config for csv metadata

* Add start of new import csv screen

* Temporary fix for compilation issue on Linux/Mac

* Disable jest bazel action for import-csv

Jest fails with an error code if no tests are available, but this would
not be noticable on Windows as Jest is not run there.

* Fix field mapping issue

* Revert "Temporary fix for compilation issue on Linux/Mac"

This reverts commit 21f8a261408cdae49ec031aa21a1b659c4f66d82.

* Add HtmlSwitch and move Switch to components

* Fix spacing and make selectors consistent

* Fix shortcut tooltip

* Place import button at the top with path

* Fix meta column indices

* Remove NotetypeForString

* Fix queue and type of foreign cards

* Support different dupe resolution strategies

* Allow dupe resolution selection when importing CSV

* Test import of unnormalized text

Close  #1863.

* Fix logging of foreign notes

* Implement CSV exports

* Use db_scalar() in notes_table_len()

* Rework CSV metadata

- Notetypes and decks are either defined by a global id or by a column.
- If a notetype id is provided, its field map must also be specified.
- If a notetype column is provided, fields are now mapped by index
instead of name at import time. So the first non-meta column is used for
the first field of every note, regardless of notetype. This makes
importing easier and should improve compatiblity with files without a
notetype column.
- Ensure first field can be mapped to a column.
- Meta columns must be defined as `#[meta name]:[column index]` instead
of in the `#columns` tag.
- Column labels contain the raw names defined by the file and must be
prettified by the frontend.

* Adjust frontend to new backend column mapping

* Add force flags for is_html and delimiter

* Detect if CSV is HTML by field content

* Update dupe resolution labels

* Simplify selectors

* Fix coalescence of oneofs in TS

* Disable meta columns from selection

Plus a lot of refactoring.

* Make import button stick to the bottom

* Write delimiter and html flag into csv

* Refetch field map after notetype change

* Fix log labels for csv import

* Log notes whose deck/notetype was missing

* Fix hiding of empty log queues

* Implement adding tags to all notes of a csv

* Fix dupe resolution not being set in log

* Implement adding tags to updated notes of a csv

* Check first note field is not empty

* Temporary fix for build on Linux/Mac

* Fix inverted html check (dae)

* Remove unused ftl string

* Delimiter → Separator

* Remove commented-out line

* Don't accept .json files

* Tweak tag ftl strings

* Remove redundant blur call

* Strip sound and add spaces in csv export

* Export HTML by default

* Fix unset deck in Mnemosyne import

Also accept both numbers and strings for notetypes and decks in JSON.

* Make DupeResolution::Update the default

* Fix missing dot in extension

* Make column indices 1-based

* Remove StickContainer from TagEditor

Fixes line breaking, border and z index on ImportCsvPage.

* Assign different key combos to tag editors

* Log all updated duplicates

Add a log field for the true number of found notes.

* Show identical notes as skipped

* Split tag-editor into separate ts module (dae)

* Add progress for CSV export

* Add progress for text import

* Tidy-ups after tag-editor split (dae)

- import-csv no longer depends on editor
- remove some commented lines
2022-06-01 20:26:16 +10:00
RumovZ
9c54f85be6
Import submodules directly (#1662)
* Use submodule imports in aqt

* Use submodule imports in pylib

* More submodule imports in pylib

These required removing some direct imports to get rid of import cycles.
2022-02-13 13:40:47 +10:00
Henrik Giesel
a8d4774cdb
Add _raw methods for all methods in the backend (#1594)
* Add _bytes methods for all methods in the backend

Expose get_note in qt/aqt/mediasrv.py

* Satisfy formatter

* Rename _bytes function to _raw and have them bytes as input

* Fix backend generation

* Use lib/proto/deckOptions in deck-options

* Add exposed_backend to qt/aqt/mediasrv.py

* Move some more backend methods to exposed_backend_list

* Use protobufjs for congrats and i18n

* Use protobufjs for completeTag

* Use protobufjs services in change-notetype

* Reorder post handlers in alphabetical manner

* Satisfy tests

* Remove unused collection methods

* Rename access_backend to raw_backend_request

* Use _vendor.stringcase instead of creating a new function

* Remove SKIP_UNROLL_OUTPUT

* Directly call _run_command in non _raw methods

* Remove TranslateString, ChangeNotetype and CompleteTag from SKIP_UNROLL_INPUT

* Remove UpdateDeckConfigs from SKIP_UNROLL_INPUT

* Remove ChangeNotetype from SKIP_UNROLL_INPUT

* Remove SKIP_UNROLL_INPUT

* Fix typing issue with translate_string

- Adds typing support for Protobuf maps in genbackend.py

* Do not emit convenience method for protobuf TranslateString
2022-01-21 21:32:39 +10:00
Damien Elmes
972c9da12e don't cache js files
https://forums.ankiweb.net/t/anki-2-1-50-beta/15608/101
2021-12-28 13:50:11 +10:00
Hikaru Y
e92d7f13e3
Preload external css files to prevent flash of unstyled content (#1558)
* Preload external css files to prevent flash of unstyled content

This is an implementation of the approach mentioned in the commit
message of 46b85d5.

* Tweak max_age value for css files

Ensure that css preloading works even on a slow PC.
2021-12-16 21:47:10 +10:00
Damien Elmes
46b85d508a fix flicker when external CSS used
When we updated to flask 2.0, the default caching time changed to
0. When setting the HTML of a new card side in the DOM, the browser
first removes the existing content (including styling), then sends a
HTTP request to us to check whether the file has changed or not. By the
time the answer has arrived, the browser has repainted without the
styling, and thus we get a flicker.

A side-effect of reverting to flask 1.x behaviour is that external changes
to media files will not be reflected in Anki for an hour, unless Anki
is restarted, or the caches are cleared manually with an add-on. An
alternative approach would be to pre-fetch the css files like we do with
images, but there are other things like fonts to think about as well.

Closes #1455
2021-12-14 12:06:16 +10:00
Damien Elmes
52f17c12e0 Partially revert "Fix issue 1362 and add a default favicon.ico (#1369)"
Kept the favicon, but have reverted the rest, as it unfortunately did
not seem to prevent the issue from occurring.

Original discussion: https://github.com/ankitects/anki/pull/1369

This reverts commit 6d0f7e7f05.
2021-12-08 21:20:39 +10:00
Damien Elmes
ebad6ad379 update platform checks (eg isWin -> is_win) + devMode 2021-11-25 09:06:16 +10:00
Damien Elmes
675155e025 minor wording tweaks and a docstring 2021-11-23 12:18:50 +10:00
evandrocoan
6d0f7e7f05
Fix issue 1362 and add a default favicon.ico (#1369) 2021-11-23 12:18:32 +10:00
Damien Elmes
7a6acc83a5 when a post handler fails, log the traceback to the console
https://forums.ankiweb.net/t/cant-change-note-type/14809
2021-11-12 11:54:13 +10:00
Damien Elmes
5a8e064a7d updated package scripts 2021-10-28 18:46:45 +10:00
Damien Elmes
b5e9eba26f use ResourceReader for serving bundled web files 2021-10-28 18:31:12 +10:00
Damien Elmes
e8c5802a96 pass sole arg to cardStats as a dictionary
Easier to extend in the future, or (de)serialize in a strongly-typed
language.
2021-10-14 19:28:51 +10:00
RumovZ
3672b0fe73
Switch CardInfoDialog to ts page (#1414)
* Only collect card stats on the backend ...

... instead of rendering an HTML string using askama.

* Add ts page Card Info

* Update test for new `col.card_stats()`

* Remove obsolete CardStats code

* Use new ts page in `CardInfoDialog`

* Align start and end instead of left and right

Curiously, `text-align: start` does not work for `th` tags if assigned
via classes.

* Adopt ts refactorings after rebase

#1405 and #1409

* Clean up `ts/card-info/BUILD.bazel`

* Port card info logic from Rust to TS

* Move repeated field to the top

https://github.com/ankitects/anki/pull/1414#discussion_r725402730

* Convert pseudo classes to interfaces

* CardInfoPage -> CardInfo

* Make revlog in card info optional

* Add legacy support for old card stats

* Check for undefined instead of falsy

* Make Revlog separate component

* drop askama dependency (dae)

* Fix nightmode for legacy card stats
2021-10-14 19:22:47 +10:00
Damien Elmes
d9bff1e8ce provide root_path to flask
Flask tries to look up root_path using __file__, which may not be
available. It's used for static file hosting and templates, neither
of which we use.
2021-10-12 16:27:57 +10:00
Damien Elmes
d33c66e195 change setHtml() to serve content via media server
- fixes https://forums.ankiweb.net/t/deck-list-is-blank/2241/2
- fixes the security warnings on Qt 6, by ensuring our pages and
resources are coming from the same origin
2021-10-12 16:27:03 +10:00
Damien Elmes
4b5004c472 refactor mediasrv request processing
_redirectWebExports was doing more than it was originally intended for,
and it was difficult to follow.
2021-10-12 16:22:09 +10:00
Damien Elmes
b9251290ca run pyupgrade over codebase [python upgrade required]
This adds Python 3.9 and 3.10 typing syntax to files that import
attributions from __future___. Python 3.9 should be able to cope with
the 3.10 syntax, but Python 3.8 will no longer work.

On Windows/Mac, install the latest Python 3.9 version from python.org.
There are currently no orjson wheels for Python 3.10 on Windows/Mac,
which will break the build unless you have Rust installed separately.

On Linux, modern distros should have Python 3.9 available already. If
you're on an older distro, you'll need to build Python from source first.
2021-10-04 15:05:48 +10:00
Damien Elmes
4da1c77220 add basic tag completion to backend
Matches should arrive in alphabetical order. Currently results are not
capped (JS should be able to handle ~1k tags without too much hassle),
and no reordering based on match location is done. Matches are substring
based, and multiple can be provided, eg "foo::bar" will match
"foof::baz::abbar".

This is not hooked up properly on the frontend at the moment -
updateSuggestions() seems to be missing the most recently typed character,
and is not updating the list of completions half the time.
2021-09-09 15:38:08 +02:00
Damien Elmes
c6c9721c53 update congrats screen periodically; automatically move back to study 2021-08-02 16:05:18 +10:00
Damien Elmes
61e86cc29d new change notetype implementation for the frontend
- changes can now be undone
- the same field can now be mapped to multiple target fields, allowing
fields to be cloned
- the old Qt dialog has been removed
- the old col.models.change() API calls the new code, to avoid
breaking existing consumers. It requires the field map to always
be passed in, but that appears to have been the common case.
- closes #1175
2021-06-10 22:19:24 +10:00
Damien Elmes
581480191a show note about old options screen if any add-ons are installed 2021-05-19 16:20:25 +10:00
Damien Elmes
9edac805ad support card state mutator in test scheduler
Documentation to come
2021-05-17 16:59:02 +10:00
Damien Elmes
30f5269304 hook new deck config screen up behind an env var 2021-04-22 10:59:16 +10:00
Damien Elmes
55277aa90a implement deck config saving on JS end 2021-04-20 19:50:05 +10:00
Damien Elmes
76eb119870 add schema change prompt to removal, tweak return struct 2021-04-18 17:33:12 +10:00
Damien Elmes
6632b63a11 support overriding listening IP during dev 2021-04-16 10:23:13 +10:00
Damien Elmes
0b8ae7225a add a shortcut to speed up local ts dev
@hgiesel if you're not already doing something like this, you might
find this useful
2021-04-14 23:21:33 +10:00
Damien Elmes
7f738c11a2 deck config prototype work in progress
Still in the early stages, and not hooked up yet.
2021-04-14 22:33:10 +10:00
Damien Elmes
b57e9be46f allow js to request specific i18n modules
Brings the payload on the congrats page with a non-English language
down from about 150k to 15k
2021-03-26 21:43:36 +10:00
Damien Elmes
88c002f4eb convert qt strings to f-strings with flynt
Also revealed an incorrect type def in editor.py that mypy wasn't
noticing before :-(
2021-02-11 10:09:06 +10:00
Damien Elmes
bb29ce88f3 minor code cleanups with pyupgrade
- pyupgrade --py38-plus --keep-runtime-typing --keep-percent-format
- third-party mpv and winpaths excluded
2021-02-11 09:43:40 +10:00
Damien Elmes
4a5ef69068 add remaining types and disable missing types on (almost) all aqt 2021-02-03 00:00:29 +10:00
Damien Elmes
a56b09b987 add a bunch of return types 2021-02-01 23:53:23 +10:00
Damien Elmes
f15715fb07 add types to various other files
Mainly automated with MonkeyType
2021-02-01 22:08:56 +10:00
Damien Elmes
6c483bb577 add public wrappers for remaining backend functions 2021-01-31 18:56:16 +10:00
Damien Elmes
9d853bbb03 start work on more clearly defining backend/protobuf boundaries
- anki._backend stores the protobuf files and rsbackend.py code
- pylib modules import protobuf messages directly from the
_pb2 files, and explicitly export any will be returned or consumed
by public pylib functions, so that calling code can import from pylib
- the "rsbackend" no longer imports and re-exports protobuf messages
- pylib can just consume them directly.
- move errors to errors.py

Still todo:

- rsbridge
- finishing the work on rsbackend, and check what we need to add
back to the original file location to avoid breaking add-ons
2021-01-31 18:55:45 +10:00
Damien Elmes
4ac9ad1407 show actual error when graphData fails
The original reason for the catch-all message was users with bad
data such as decimal intervals, but those get automatically coerced
these days. The common case should now be invalid search strings, which
we can show verbatim.
2021-01-29 14:39:05 +10:00
Damien Elmes
37ca8afaf6 minor wording tweak: GraphsPreferences -> GraphPreferences 2021-01-23 20:47:45 +10:00
Henrik Giesel
bf130d1da0 Change the function name in genbackend.py 2021-01-22 20:05:28 +01:00
Henrik Giesel
de71123ab4 Rename GraphsPreferencesOut to simply GraphsPreferences 2021-01-22 19:39:03 +01:00
Henrik Giesel
d1ada88657 Successfully send post request to rust 2021-01-22 14:37:24 +01:00
Henrik Giesel
054c30a695 Add non-functioning logic for settings graphs preferences 2021-01-22 13:14:35 +01:00
Henrik Giesel
665a13e378 Add GraphsPreferences endpoint to backend 2021-01-22 13:13:48 +01:00
Henrik Giesel
ffc40d8959 Replace browsersel with css-browser-selector in aqt files and add redirect 2020-12-31 16:47:20 +01:00
Henrik Giesel
ed8b12e1d6 Add redirect for jquery-ui 2020-12-30 12:08:12 +01:00
Damien Elmes
9e1eaacc06 Revert "Merge pull request #873 from hgiesel/otherjsdeps"
This reverts commit 62600051ae, reversing
changes made to 88553acb0d.

- Standard graphs render incorrectly on latest version - the wrong number
of days are shown, and the grid lines look wrong. Any version after 0.8.3
seems to suffer from this problem.
- Pie graphs and stack graphs don't render - they are provided in separate
files, and plot.js in previous Anki versions has them included in the one
file. To maintain compatibility with add-ons, we'd need to create a single
file as before, instead of importing multiple files.

If the above issues are fixed I'd be happy to merge this in again, but
as the old graphs are on the way out, it's probably not worth the effort.
2020-12-30 14:13:52 +10:00
Henrik Giesel
a6319326a9 Redirect references to old plotting library version 2020-12-29 12:15:46 +01:00