Skip to the content ForoHackers
Log inJoin

The accent marks are identical until my tiny corpus counts them

Data engineeringInformation retrievalNatural language processing
#16774 · · Quote · Report

remember @LeSobraUnOjal arguing about café in the index last year? my corpus has the name Inés twice. one transcription uses é; the other e followed by a combining acute. search should find both. editor wants the original spellings left alone, especially since she'll be the one people blame when I break them.

#16900 · · Quote · Report

اول متن اصلی را نگه دار. اگر برای جستجو کلید دیگری می‌سازی، نباید بعداً مجبور شوی از روی کلید حدس بزنی نویسنده چه نوشته بود.

#16901 · · Quote · Report

Raw corpus: two distinct sequences. NFC search key: one key, two occurrences. Now if I highlight the first four code points of the original spelling because the key has length four, one transcription loses its final s under the highlight.

#16902 · · Quote · Report

che, este Inés me costó un café. guardá una referencia al tramo de la transcripción original para cada coincidencia. no le impongas al original los cuatro caracteres del otro tramo. mi sí y mi si siguen siendo dos palabras con cosas distintas que decir.

#16903 · · Quote · Report

We kept both source ranges. Printed each original line with its own full name highlighted. Two hits, no missing s, no published transcript rewritten. I'd buy the editor a bun for never letting us call that enterprise accent policy.

#16904 · · Quote · Report

برای نمونهٔ دوم هم همان سطر اصلی را نشان دادید؟ چهار تا حرف روی صفحه هست، ولی کلیدِ چهارخانه نباید اندازهٔ متنِ اصلی را تعیین کند.

#16905 · · Quote · Report

yep. both original lines, same display face, select the whole name in each. only the copy of the decomposed one still has five code points. our editor requested a print of both and wrote EXACTLY at the top, in red, like a mark I once got in school.

You are not logged in, so this goes up anonymously: no account, no name and no address is stored with it, and it may carry no link and no image. Log in or join to post under your name. Our web server's logs keep the address and time of every request for up to 14 days; to keep yours out of them, use Tor. To reach the staff in confidence, use the drop. Sign as a returning source (a key kept only in this browser)

Sending takes a few seconds here: your browser first works out a proof of work against spam.

Reply