K-repeating Substrings: a String-Algorithmic Approach to Privacy-Preserving Publishing of Textual...

K-repeating substrings:a string-algorithmic approach to privacy-preserving publication of texual data

2014-12-14, PACLIC 28 (Phuket)

Yusuke Matsubara*Kôiti Hasida*

*The University of Tokyo

Background: easier sharing of sensitive data with less labor

• Electronic health records contains sensitive information about individual patients. Removing sensitive information allows– easier data sharing between institutions or

between researchers and institutions– while protecting patients from potential privacy

violation

but requires high-cost human labor to annotate manually

Task: Finding and hiding sensitive expression from text

• Patient names• Hospital

names• Some of the

disease names• Relevant

Task: Finding and hiding sensitive expression from text

• Patient names• Hospital

names• Some of the

disease names• Relevant

Previous work

Supervised learning & heuristic patterns• i2b2 challenge

(Uzuner et al., 2007)

PPDP for unstructured data (Liu, 2012)• Words or bags-of-

words as data values

Requires large training data,available in limited languages

Not fit to free-form text

Frequency may inform sensitivity (to some extent)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Chances of being "sensitive" for a token with a certain frequency

from i2b2 deid training set PHI Non PHI

Simple word frequency is not enough

• Common words may form a rare sequence, denoting a highly specific entity– “PINE STREET 10, SAN FRANCISCO, CA”

• Not trivial for morphologically complex languages/terminologies (using simple space delimiters may not be best)

Goal of this work

• Distinguishing frequent/rare substrings efficiently which works as an alternative to word frequency

Method

Overview: two-step approach

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Finding maximum repeats with at least k occurrences

2. Suppressing regions not covered by non-overlapping, non-neighboring tiling

Finding repeats: is there a span covered by no repeat?a b r a c a d a r a d a bb r

Finding repeats: naively, examine all substrings (O(n2))a b r a c a d a r a d a bb

Finding repeats: only maximal ones are meaningfula b r a c a d a r a d a bb

Finding repeats: higher threshold for “repeats”a b r a c a d a r a d a bb

Finding repeats

a b r a c a d a r a d a bbabr

How can we avoid the O(n2) enumerationand still find maximal repeats?

Maximum repeats (k=2) [Ilie+Smyth 2011]

1512073

0034115023003

$abr$abracadabradabr$abradabr$acabdarabr$adabr$adabradabr$br$bracadabradabr$bradabr$cadabradabr$dab$dabradabr$

racadabradabr$radabr$

SA LCP T[SA[i]...]

T=abracadabradabr$

14 0 r$

(1) construct the suffix array and longest-common-prefix array of a given text (linear time)

1512073

0034115023003

SA LCP

-------------

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b

(2) construct an array indicating repeating spans (linear time)

1512073

0034115023003

SA LCP

-------------

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

1512073

0014115023003

SA LCP

---0---------

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b

1512073

0014115023003

SA LCP

---0------7--

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b-

1512073

0014115023003

SA LCP

---0------7--

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b

No update

1512073

0014115023003

SA LCP

---0------7--

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b

1512073

0014115023003

SA LCP

---0------5--

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 -r$ 13 b

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0------5--

No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0------5--

No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0------5--

No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0------5--

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0------5--

-No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0----6-5--

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0----6-5--

-No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0----6-5--

-No update

1512073

0014115023003

SA LCP

0123456789

101112

T[SA[i]...]

T=abracadabradabr$

abracadabrada

14 0 r$ 13 b

---0----6-5--

0123456789

101112

T=abracadabradabr$

abracadabrada

---0----6-5--

a b r a c a d a r a d a bb r

Maximum repeats (k=3, 4, ...) [proposed method]

1411131070358146

000011110000002

$ra$a$ara$abrara$abracadabrara$acadabrara$adabrara$brara$bracadabrara$cadabrara$dabrara$ra$rara$racadabrara$

SA gLCP

---------------

0123456789

1011121314

T[SA[i]...]

k=3 T=abracadabrara$

abracadabrara$

1411131070358146

000011110000002

SA gLCP

-------------

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

----------

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

-------7--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0------7--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0--3---7--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0--3-5-7--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0--3-5-7--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0--3-5-7--9--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

1411131070358146

000011110000002

SA gLCP

0--2-5-7--9--

0123456789

1011121314

T[SA[i]...]

abracadabrara$

Greedily tiling with the frequent-enough substrings

• Check from the longest ones– If it overlaps or neighbors with none of

accepted repeats, accept it

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

• Output the spans that are not covered by any repeats

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

• Output the spans that are not covered by any repeats

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Experiments

Experimental setups

• The i2b2 de-identification dataset (DEID)– 1.2 M bytes (227k words) [Uzuner et al 2007]– Gold standard annotated with PHI by experts– To show the effectiveness in identifying sensitive

information

• Japanese Wiktionary (WKT)– 45 M bytes– Contains long repetitive XML elements– To show the scalability

Experimental setups: DEID

• Baselines (WORD)– Word frequency (with different thresholds)

• Proposed method 1 (MR)– Different k– Each repeat contains at least 6 characters– Words with more than 20% hidden are considered

completely hidden

• Proposed method 2 (MR+WORD)– Logical AND of WORD and MR on suppression

DEID: recall-precision

Time: linear-time computation in practice (WKT)

Future work

• Combining with supervised methods• More space efficiency• Document (record) frequency • Other applications

– Term extraction– Unsupervised word segmentation– Near-duplicate detection

Conclusions

• A suffix-array based method to detect rare sequences in text• As cheap as and more effective than simple

word frequency for text de-identification• Linear-time computation in theory and in

practice

Future work (ongoing)

• Comparison against N-grams– Preview: higher-order ones (N=8, 9, 10, ...)

seem to be comparable, but with a higher computational cost

• Smarter optimization than greedy– Preview: dynamic-programming optimization

is possible, but gives marginal improvement only

Source code

https://github.com/whym/growthring

K-repeating Substrings: a String-Algorithmic Approach to Privacy-Preserving Publishing of Textual...

Technology

Transcript of K-repeating Substrings: a String-Algorithmic Approach to Privacy-Preserving Publishing of Textual...

Repeating Figures and Symmetries How can tessellations be made with repeating figures? What four types of symmetry do you look for in repeating figures?

Efficient Algorithms for Locating Maximum Average Consecutive Substrings

Algorithmic marketplace

Repeating Successes

Create Repeating Patterns

Algorithmic Reverberation - Arqen.comarqen.com/wp-content/docs/Hybrid-Convolution-Algorithmic-Reverb.pdf · DSP Project. Algorithmic Reverberation . Combining Moorer’s reverberator

Lies, Damned Lies, and Substrings

Multiprocessor computer architectures : algorithmic design ... · Multiprocessor computer architectures : algorithmic design and ... ARCHITECTURES: ALGORITHMIC DESIGN AND APPLICATIONS

From Algorithmic States to Algorithmic Brainswhitakerinstitute.ie/wp-content/uploads/2014/02/... · From Algorithmic States to Algorithmic Brains The modern age is characterised by

Repeating Lenin

Road Signs “Some Things Bear Repeating” Repeating”

Algorithmic research

Finding substrings my $sequence = "gatgcaggctcgctagcggct"; #Does this string contain a startcodon? if…

Junos® OS Frame Relay Feature Guide for Routing Devices · NOTE:TheIQEPICssupportonlythefollowingalgorithms: all-ones-repeating Repeating one bits all-zeros-repeating Repeating zero

Mathematical Recipes for Never-repeating Quasicrystals · Mathematical Recipes for Never-repeating Quasicrystals ... ematical recipes for never-repeating quasicrystals. Repeating

Efï¬cient Enumeration of Phylogenetically Informative Substrings

Busqueda de Substrings

Loving Repeating

Finding Characteristic Substrings from Compressed Texts

Algorithmic Journalism and Its Impacts on Work · algorithmic news, and to define the algorithms that generate algorithmic news, we will use the term algorithmic journalists. Algorithmic