-
Notifications
You must be signed in to change notification settings - Fork 5
/
srfi-135.html
2983 lines (2648 loc) · 110 KB
/
srfi-135.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html public '-//W3C//DTD HTML 4.01//EN'
'http://www.w3.org/TR/REC-html4/strict.dtd'>
<!-- HTML skeleton (including style hackery) copied from srfi-130.html -->
<!-- Can I have bangs, plusses, or slashes in #tags? Spaces?
Yes: plus, bang, star No: space Yes: slash, question, ampersand
You can't put sharp in a path, so anything goes, really.
Nonetheless, some of these confuse Netscape, so I'll avoid them.
-->
<!--========================================================================-->
<html>
<head>
<meta name="keywords" content="Scheme, programming language, strings, texts, Unicode, SRFI" />
<link rev=made href="mailto:[email protected]" />
<title>SRFI 135: Immutable Texts</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="stylesheet" href="/srfi.css" type="text/css" />
<!-- Should have a media=all to get, for example, printing to work.
== But my Netscape will completely ignore the tag if I do that.
-->
<style type="text/css">
/* A little general layout hackery for headers & the title. */
body { margin-left: +7%;
font-family: "Helvetica", sans-serif;
}
/* Netscape workaround: */
td, th { font-family: "Helvetica", sans-serif; }
code, pre { font-family: "courier new", "courier"; }
div.inset { margin-left: +5%; }
h1 { margin-left: -5%; }
h1, h2 { clear: both; }
h1, h2, h3, h4, h5, h6 { color: blue }
div.title-text { font-size: large; font-weight: bold; }
h3 { margin-top: 2em; margin-bottom: 0em }
/* "Continue" class marks text that isn't really the start
** of a new paragraph — e.g., continuing a para after a
** code sample.
*/
p.continue { text-indent: 0em; margin-top: 0em}
div.indent { margin-left: 2em; } /* General indentation */
pre.code-example { margin-left: 2em; } /* Indent code examples. */
/* This stuff is for definition lists of defined procedures.
** A proc-def1 is used when you want a stack of procs to go
** with one dd body. In this case, make the first
** proc a proc-def1, following ones proc-defi's, and the last one
** a proc-defn.
**
** Unfortunately, Netscape has huge bugs with respect to style
** sheets and dl list rendering. We have to set truly random
** values here to get the rendering to come out. The proper values
** are in the following style sheet, for Internet Explorer.
** In the following settings, the *comments* say what the
** setting *really* causes Netscape to do.
**
** Ugh. Professional coders sacrifice their self-respect,
** that others may live.
*/
/* m-t ignored; m-b sets top margin space. */
dt.proc-def1 { margin-top: 0ex; margin-bottom: 3ex; }
dt.proc-defi { margin-top: 0ex; margin-bottom: 0ex; }
dt.proc-defn { margin-top: 0ex; margin-bottom: 0ex; }
/* m-t works weird depending on whether or not the last line
** of the previous entry was a pre. Set to zero.
*/
dt.proc-def { margin-top: 0ex; margin-bottom: 3ex; }
/* m-b sets space between dd & dt; m-t ignored. */
dd.proc-def { margin-bottom: 0.5ex; margin-top: 0ex; }
/* Boldface the name of a procedure when it's being defined. */
code.proc-def { font-weight: bold; font-size: 110%}
/* For the index of procedures.
** Same hackery as for dt.proc-def, above.
*/
/* m-b sets space between dd & dt; m-t ignored. */
dd.proc-index { margin-bottom: 0ex; margin-top: 0ex; }
/* What the fuck? */
pre.proc-index { margin-top: -2ex; }
/* Pull the table of contents back flush with the margin.
** Both NS & IE screw this up in different ways.
*/
#toc-table { margin-top: -2ex; margin-left: -5%; }
/* R5RS proc names are in italic; extended R5RS names
** in italic boldface.
*/
span.r5rs-proc { font-weight: bold; }
span.r5rs-procx { font-style: italic; font-weight: bold; }
/* Spread out bibliographic lists. */
/* More Netscape-specific lossage; see the following stylesheet
** for the proper values (used by IE).
*/
dt.biblio { margin-bottom: 3ex; }
/* Links to draft copies (e.g., not at the official SRFI site)
** are colored in red, so people will use them during the
** development process and kill them when the document's done.
*/
a.draft { color: red; }
</style>
<style type="text/css" media=all>
/* Nastiness: Here, I'm using a bug to work around a bug.
** Netscape rendering bugs mean you need bogus <dt> and <dd>
** margin settings — settings which screw up IE's proper rendering.
** Fortunately, Netscape has *another* bug: it will ignore this
** media=all style sheet. So I am placing the (proper) IE values
** here. Perhaps, one day, when these rendering bugs are fixed,
** this gross hackery can be removed.
*/
dt.proc-def1 { margin-top: 3ex; margin-bottom: 0ex; }
dt.proc-defi { margin-top: 0ex; margin-bottom: 0ex; }
dt.proc-defn { margin-top: 0ex; margin-bottom: 0.5ex; }
dt.proc-def { margin-top: 3ex; margin-bottom: 0.5ex; }
pre { margin-top: 1ex; }
dd.proc-def { margin-bottom: 2ex; margin-top: 0.5ex; }
/* For the index of procedures.
** Same hackery as for dt.proc-def, above.
*/
dd.proc-index { margin-top: 0ex; }
pre.proc-index { margin-top: 0ex; }
/* Spread out bibliographic lists. */
dt.biblio { margin-top: 3ex; margin-bottom: 0ex; }
dd.biblio { margin-bottom: 1ex; }
</style>
<style type="text/css" media="all">
/* Added by Will Clinger so lists don't look so crowded. */
ul li { margin-top: 2pt; margin-bottom: 2pt; }
</style>
</head>
<body>
<!--========================================================================-->
<H1>Title</H1>
<div class=title-text>Immutable Texts</div>
<!--========================================================================-->
<H1>Author</H1>
William D Clinger
<H1>Status</H1>
<p>This SRFI is currently in <em>final</em> status. Here is <a href="https://srfi.schemers.org/srfi-process.html">an explanation</a> of each status that a SRFI can hold. To provide input on this SRFI, please send email to <code><a href="mailto:srfi+minus+135+at+srfi+dotschemers+dot+org">srfi-135@<span class="antispam">nospam</span>srfi.schemers.org</a></code>. To subscribe to the list, follow <a href="http://srfi.schemers.org/srfi-list-subscribe.html">these instructions</a>. You can access previous messages via the mailing list <a href="https://srfi-email.schemers.org/srfi-135">archive</a>.</p>
<ul>
<li>Received: 2016-06-06</li>
<li>60-day deadline: 2016-08-05</li>
<li>Draft #1 published: 2016-06-06</li>
<li>Draft #2 published: 2016-06-11</li>
<li>Draft #3 published: 2016-06-17</li>
<li>Draft #4 published: 2016-07-09</li>
<li>Finalized: 2016-09-06</li>
<li>Revised to fix errata:
<ul>
<li>2024-09-02 (Revised to fix missing argument in <a href="#textual-fold">examples</a> of <code>textual-fold</code> and <code>textual-fold-right</code>.)</li></ul></li>
</ul>
<h1>Table of contents</h1>
<ul id=toc-table>
<li><a href="#Abstract">Abstract</a></li>
<li><a href="#Issues">Issues</a></li>
<li><a href="#ProcedureIndex">Procedure index</a></li>
<li><a href="#Rationale">Rationale</a></li>
<li><a href="#Specification">Specification</a>
<ul>
<li><a href="#Concepts">Concepts</a>
<ul>
<li><a href="#LibraryName">Name of library</a></li>
<li><a href="#ConceptualModel">Conceptual model</a></li>
<li><a href="#Subtypes">Subtypes</a></li>
<li><a href="#ExternalRepresentation">External representation</a></li>
<li><a href="#TextualPorts">Textual input and output ports</a></li>
<li><a href="#SharedStorage">Shared storage</a></li>
<li><a href="#NamingConventions">Naming conventions</a></li>
<li><a href="#PerformanceRequirements">Performance requirements</a></li>
<li><a href="#Unicode">Unicode</a></li>
</ul></li></ul></li>
<li><a href="#Notation">Notation</a></li>
<li><a href="#Procedures">Procedures</a>
<ul>
<li><a href="#Predicates">Predicates</a></li>
<li><a href="#Constructors">Constructors</a></li>
<li><a href="#Conversion">Conversion</a></li>
<li><a href="#Selection">Selection</a></li>
<li><a href="#Replacement">Replacement</a></li>
<li><a href="#Comparison">Comparison</a></li>
<li><a href="#PrefixesSuffixes">Prefixes & suffixes</a></li>
<li><a href="#Searching">Searching</a></li>
<li><a href="#CaseConversion">Case conversion</a></li>
<li><a href="#Concatenation">Concatenation</a></li>
<li><a href="#FoldMap">Fold & map & friends</a></li>
<li><a href="#ReplicationSplitting">Replication & splitting </a></li>
</ul>
</li>
<li><a href="#SampleImp">Sample implementations</a></li>
<li><a href="#Acknowledgements">Acknowledgements</a></li>
<li><a href="#Links">References & Links</a></li>
<li><a href="#Copyright">Copyright</a></li>
</ul>
<!--========================================================================-->
<h1><a name="Abstract">Abstract</a></H1>
<p>
In Scheme, strings are a mutable data type.
Although it "is an error"
(<abbr title="Revised<sup>5</sup> Report on Scheme"><a
href="#R5RS">R5RS</a></abbr>
and
<abbr title="Revised<sup>7</sup> Report on Scheme"><a
href="#R7RS">R7RS</a></abbr>)
to use <code>string-set!</code>
on literal strings or on strings returned by <code>symbol->string</code>,
and any attempt to do so "should raise an exception"
(<abbr title="Revised<sup>6</sup> Report on Scheme"><a
href="#R6RS">R6RS</a></abbr>),
all other strings are mutable.
</p>
<p>
Although many mutable strings are never actually mutated, the mere
possibility of mutation complicates specifications of libraries that
use strings, encourages precautionary copying of strings, and precludes
structure sharing that could otherwise be used to make procedures such
as <code>substring</code> and <code>string-append</code> faster and
more space-efficient.
</p>
<p>
This
<abbr title="Scheme Request for Implementation"><a
href="#SRFI">SRFI</a></abbr>
specifies a new data type of immutable texts.
It comes with efficient and portable sample implementations
that guarantee O(1) indexing
for both sequential and random access, even in systems whose
<code>string-ref</code> procedure takes linear time.
</p>
<p>
The operations of this new data type include analogues for all
of the non-mutating operations on strings specified by
the R7RS and most of those specified by
<abbr title="String cursors"><a href="#SRFI-130">SRFI 130</a></abbr>,
but the immutability of texts and
uniformity of character-based indexing simplify the
specification of those operations while avoiding several
inefficiencies associated with the mutability of Scheme's
strings.
</p>
<h1><a name="Issues">Issues</a></h1>
<p>
None.
</p>
<!--========================================================================-->
<h1><a name="ProcedureIndex">Procedure Index</a></h1>
<p>
Here is a list of the procedures provided by this SRFI:
<div class=indent>
<dl>
<dt class="proc-index"> Predicates</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#text-p">text?</a> <a href="#textual-p">textual?</a>
<a href="#textual-null-p">textual-null?</a>
<a href="#textual-every">textual-every</a> <a href="#textual-any">textual-any</a>
</pre>
</dd>
<dt class="proc-index"> Constructors</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#make-text">make-text</a> <a href="#text">text</a>
<a href="#text-tabulate">text-tabulate</a>
<a href="#text-unfold">text-unfold</a> <a href="#text-unfold-right">text-unfold-right</a>
</pre>
</dd>
<dt class="proc-index"> Conversion</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual2text">textual->text</a>
<a href="#text2string">textual->string</a> <a href="#text2vector">textual->vector</a> <a href="#text2list">textual->list</a>
<a href="#string2text">string->text</a> <a href="#vector2text">vector->text</a> <a href="#list2text">list->text</a>
<a href="#reverse-list2text">reverse-list->text</a>
<a href="#text2utf8">textual->utf8</a> <a href="#text2utf16be">textual->utf16be</a>
<a href="#text2utf16">textual->utf16</a> <a href="#text2utf16le">textual->utf16le</a>
<a href="#utf82text">utf8->text</a> <a href="#utf16be2text">utf16be->text</a>
<a href="#utf162text">utf16->text</a> <a href="#utf16le2text">utf16le->text</a>
</pre>
</dd>
<dt class="proc-index"> Selection</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#text-length">text-length</a> <a href="#textual-length">textual-length</a>
<a href="#text-ref">text-ref</a> <a href="#textual-ref">textual-ref</a>
<a href="#subtext">subtext</a> <a href="#subtextual">subtextual</a>
<a href="#textual-copy">textual-copy</a>
<a href="#textual-take">textual-take</a> <a href="#textual-take-right">textual-take-right</a>
<a href="#textual-drop">textual-drop</a> <a href="#textual-drop-right">textual-drop-right</a>
<a href="#textual-pad">textual-pad</a> <a href="#textual-pad-right">textual-pad-right</a>
<a href="#textual-trim">textual-trim</a> <a href="#textual-trim-right">textual-trim-right</a> <a href="#textual-trim-both">textual-trim-both</a>
</pre>
</dd>
<dt class="proc-index"> Replacement</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-replace">textual-replace</a>
</pre>
</dd>
<dt class="proc-index"> Comparison</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-equal-p">textual=?</a> <a href="#textual-ci-equal-p">textual-ci=?</a>
<a href="#textual-less-p">textual<?</a> <a href="#textual-ci-less-p">textual-ci<?</a>
<a href="#textual-greater-p">textual>?</a> <a href="#textual-ci-greater-p">textual-ci>?</a>
<a href="#textual-leq-p">textual<=?</a> <a href="#textual-ci-leq-p">textual-ci<=?</a>
<a href="#textual-geq-p">textual>=?</a> <a href="#textual-ci-geq-p">textual-ci>=?</a>
</pre>
</dd>
<dt class="proc-index">Prefixes & suffixes</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-prefix-length">textual-prefix-length</a> <a href="#textual-suffix-length">textual-suffix-length</a>
<a href="#textual-prefix-p">textual-prefix?</a> <a href="#textual-suffix-p">textual-suffix?</a>
</pre>
</dd>
<dt class="proc-index">Searching</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-index">textual-index</a> <a href="#textual-index-right">textual-index-right</a>
<a href="#textual-skip">textual-skip</a> <a href="#textual-skip-right">textual-skip-right</a>
<a href="#textual-contains">textual-contains</a> <a href="#textual-contains-right">textual-contains-right</a>
</pre>
</dd>
<dt class="proc-index"> Case conversion</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-upcase">textual-upcase</a> <a href="#textual-downcase">textual-downcase</a>
<a href="#textual-foldcase">textual-foldcase</a> <a href="#textual-titlecase">textual-titlecase</a>
</pre>
</dd>
<dt class="proc-index"> Concatenation</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-append">textual-append</a> <a href="#textual-concatenate">textual-concatenate</a> <a href="#textual-concatenate-reverse">textual-concatenate-reverse</a>
<a href="#textual-join">textual-join</a>
</pre>
</dd>
<dt class="proc-index">Fold & map & friends</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-fold">textual-fold</a> <a href="#textual-fold-right">textual-fold-right</a>
<a href="#textual-map">textual-map</a> <a href="#textual-for-each">textual-for-each</a>
<a href="#textual-map-index">textual-map-index</a> <a href="#textual-for-each-index">textual-for-each-index</a>
<a href="#textual-count">textual-count</a>
<a href="#textual-filter">textual-filter</a> <a href="#textual-remove">textual-remove</a>
</pre>
</dd>
<dt class="proc-index">Replication & splitting</dt>
<dd class="proc-index">
<pre class="proc-index">
<a href="#textual-replicate">textual-replicate</a> <a href="#textual-split">textual-split</a>
</pre>
</dd>
</dl>
</div>
<!--========================================================================-->
<h1><a name="Rationale">Rationale</a></h1>
<p>
The
<abbr title="Revised<sup>6</sup> Report on Scheme: Rationale"><a
href="#R6RS-Rationale">R6RS Rationale</a></abbr>
identified problems created by the mutability of strings,
and several more problems were mentioned by SRFI 130
or came up during its discussion period:
</p>
<ul>
<li>Mutability complicates the specification of higher-order procedures
that operate on strings.</li>
<li>Mutability inhibits several compiler optimizations, including
common subexpression elimination.</li>
<li>Mutability complicates reasoning about programs that use strings.</li>
<li>Mutations invalidate the string cursors of SRFI 130.</li>
<li>Using a SRFI 130 string cursor that has been invalidated by
mutation is an error, but that error is likely to go undetected,
making programs harder to test and to debug.</li>
<li>Mutations can be expensive if strings are represented
as encapsulated UTF-8 or UTF-16.</li>
<li>Although representations based on UTF-32 provide fast referencing
as well as fast mutation, they occupy more space than representations
based on UTF-8 or UTF-16.</li>
<li>Mutations preclude sharing of substructure that could save
space while making
<code>substring</code> and <code>string-append</code> run faster.</li>
</ul>
<p>
Recognizing the first three of these problems, while acknowledging
that removing mutable strings from the language would cause
"significant compatibility problems for existing code"
<a href="#R6RS-Rationale">[R6RS-Rationale]</a>,
the R6RS standard banished
<code>string-set!</code> and <code>string-fill!</code>
to a separate <code>(rnrs mutable-strings)</code> library
in hope of discouraging and/or deprecating mutation of strings.
</p>
<p>
The R7RS restored <code>string-set!</code> and <code>string-fill!</code>
to the
<code>(scheme base)</code> library and added a new mutator,
<code>string-copy!</code>.
Waiting for some revised standard
to make strings immutable is not viable.
</p>
<p>
We can, however, add a new data type of immutable texts capable
of replacing Scheme's string data type for all applications that
do not require mutation. Immutable texts do away with the problems
listed above while offering these advantages over mutable strings:
</p>
<ul>
<li>space efficiency approaching that of UTF-8 or UTF-16</li>
<li>faster sequential access (if strings use UTF-8 or UTF-16)</li>
<li>faster random access (if strings use UTF-8 or UTF-16)</li>
<li>fast extraction of subtexts</li>
<li>faster concatenation of texts</li>
</ul>
<p>
SRFI 130 aims for the second of those advantages, but cannot
achieve that advantage in any portable implementation of its
procedures.
Furthermore SRFI 130 introduces a new data type (cursors) that is
hard to use correctly (because its error situations are likely to
go undetected, partly because many of its operations accept both
cursors and character indexes, which are allowed but not required
to be the same things).
</p>
<p>
This SRFI offers all five advantages, at the expense of introducing
a new data type (immutable texts) that can be implemented portably
and efficiently and is easy to use correctly
(partly because most of its error situations are detected by its
sample implementations, and partly because its character indexes
are the same as those used by Scheme's mutable strings).
</p>
<p>
See also the <a href="#Unicode">discussion of Unicode</a>.
</p>
<p>
This SRFI is based upon SRFI 130, copying much of its structure
and wording, which should make it easier to compare this SRFI
against SRFI 130 and to convert programs using SRFI 130 to use
immutable texts instead.
</p>
<!--========================================================================-->
<h1><a name="Specification">Specification</a></h1>
<h2 id="Concepts">Basic concepts</h2>
<h3><a name="LibraryName">Name of library</a></h3>
<p>
The procedures specified by this SRFI are exported by the
<code>(srfi 135)</code> library.
In R6RS systems that do not yet support R7RS library names,
the name of this library is <code>(srfi :135)</code>.
</p>
<p>
It is recommended, but not required, that this library also
be made available under the alternative name
<code>(srfi 135 texts)</code>. That alternative library
should export exactly the same bindings as the
<code>(srfi 135)</code> library, so libraries and programs
can import both libraries without name conflicts.
</p>
<!--========================================================================-->
<h3><a name="ConceptualModel">Conceptual model</a></h3>
<p>
Immutable texts are like strings except they can't be mutated.
</p>
<p>
Immutability makes it easier to use space-efficient
representations such as UTF-8 and UTF-16 without incurring the
cost of scanning from the beginning when character indexes are
used (as with <code>string-ref</code>).
</p>
<p>
When mutation is not needed, immutable texts are likely to be
more efficient than strings with respect to space or time.
In some implementations, immutable texts may be more efficient
than strings with respect to both space and time.
</p>
<!--========================================================================-->
<h3><a name="Subtypes">Subtypes</a></h3>
<p>
This SRFI defines two new types:
</p>
<ul>
<li>
<em>text</em> is a type consisting of the immutable texts
for which <code>text?</code> returns true.
</li>
<li>
<em>textual</em> is a union type consisting of the texts
and strings for which <code>textual?</code> returns true.
</li>
</ul>
<p>
The subtypes of the new <em>textual</em> type include
the new <em>text</em> type and
Scheme's traditional <em>string</em> type, which consists
of the values for which <code>string?</code> returns true.
The <em>string</em> type includes both mutable strings and the
(conceptually) immutable strings that are the values of string
literals and calls to <code>symbol->string</code>.
</p>
<p>
Implementations of this SRFI are free to extend the <em>textual</em>
type by adding new subtypes, provided all procedures whose names
begin with <code>textual-</code> are extended to accept values of
those new subtypes.
Implementations of this SRFI should not extend the <em>text</em>
type unless its extended values are immutable, are accepted as
texts by all procedures of this SRFI (including the <code>text?</code>
predicate), and achieve the
<a href="#PerformanceRequirements">performance required by this SRFI</a>
with respect to both time and space.
</p>
<!--========================================================================-->
<h3><a name="ExternalRepresentation">External representation</a></h3>
<p>
This SRFI does not require any particular external representation
for immutable texts, but recommends immutable texts have almost
the same external representation as strings, substituting Unicode's
left-pointing and right-pointing double angle quotation marks
(« and », code points <code>#xab</code> and <code>#xbb</code>)
for the double quotes that delimit strings,
and allowing those double angle quotation marks to be escaped
within the external representations of both texts and strings.
That external representation is used by this SRFI's examples.
</p>
<p>
When feasible, implementations of this SRFI should also consider:
</p>
<ul>
<li>extending the <code>equal?</code> procedure to regard two
immutable texts <var>t1</var> and <var>t2</var> as equal
if and only if
<code>(textual=? <var>t1</var> <var>t2</var>)</code>,
while regarding an immutable text as unequal to anything
that isn't an immutable text.</li>
<li>extending the <code>display</code> procedure to accept
immutable texts, treating them the same as strings;</li>
<li>extending the <code>write</code> procedure to generate the
external syntax recommended for immutable texts;</li>
<li>extending the <code>read</code> procedure to accept the
external syntax recommended for immutable texts;</li>
<li>extending interpreters and compilers to accept quoted
literals expressed using the external syntax recommended
for immutable texts; R7RS section 4.1.2 mandates this
extension if <code>read</code> is extended to accept
the external syntax for texts.</li>
</ul>
<p>
<i>Note:</i>
Those extensions cannot be implemented portably, so portable code
should not rely on them.
</p>
<!--========================================================================-->
<h3><a name="TextualPorts">Textual input and output ports</a></h3>
<p>
Textual input and output ports analogous to string input and
output ports would be nice, but they too cannot be implemented
portably. Leaving them for another SRFI allows all of this
SRFI to be implemented portably with reasonable efficiency.
</p>
<!--========================================================================-->
<h3><a name="SharedStorage">Shared storage</a></h3>
<p>
All strings and other mutable objects returned by the procedures
specified within this SRFI are newly allocated and may be mutated
with abandon.
</p>
<p>
No externally visible string ever shares storage with any text.
All strings and other mutable objects passed to the procedures
specified within this SRFI may be mutated without affecting the
behavior of any text.
</p>
<p>
The immutability of texts allows sharing of substructure, so
<code>subtext</code>, <code>textual-append</code>, and similar
operations can be faster and more space-efficient than Scheme's
<code>substring</code> and <code>string-append</code> operations.
</p>
<p>
Although distinct texts may share storage internally, this is
undetectable because texts are immutable and the procedures that
operate on texts do not directly expose any of their internal
components.
</p>
<p>
Implementations that share storage between texts must satisfy
the following requirement: There is some reasonably small fixed
bound on the ratio of storage used by the shared representation
divided by the storage that would be used by an unshared
representation.
</p>
<p>
<i>Example:</i>
For the
<a href="#SampleImp">sample implementations</a>
with their default configurations,
the worst case arises with UTF-8, when a 1-character ASCII
text retains up to 127 characters of a text that is no longer
reachable, and all 127 of those retained characters lie outside
Unicode's Basic Multilingual Plane (BMP).
Making reasonable assumptions about the representations of
records, vectors, bytevectors, and strings on a 64-bit machine,
that shared text would occupy no more than about 16 times the
space occupied by an unshared representation.
If the retained characters were in the BMP, the shared text
would occupy no more than about 8 times the space occupied
by an unshared representation.
If the retained characters were ASCII, the shared text would
occupy no more than about 4 times the space occupied by an
unshared representation.
The sample implementations can be configured to reduce those
worst-case bounds, most obviously by reducing the maximum
number of characters that can be shared with a very short
text.
</p>
<!--========================================================================-->
<h3><a name="NamingConventions">Naming conventions</a></h3>
<p>
The procedures of this SRFI follow
a consistent naming scheme, and are consistent with the conventions
developed in SRFI 1 and used in SRFI 13 and SRFI 130.
Indeed, most of the names specified here were derived from SRFI 130's
names by replacing <code>string</code> with <code>text</code> or
<code>textual</code>.
As in SRFI 130,
procedures that have left/right directional variants
use no suffix to specify left-to-right operation,
<code>-right</code> to specify
right-to-left operation, and <code>-both</code> to specify both.
</p>
<p>
Note, however, that <code>textual-index</code>,
<code>textual-index-right</code>,
<code>textual-skip</code>, and
<code>textual-skip-right</code>,
return <code>#f</code> when no match is found.
In SRFI 130, their analogues always return cursors.
</p>
<p>
The order of common arguments is consistent across the
different procedures.
</p>
<p>
For convenience, most procedures that accept a text as argument
will also accept a string. When given a string, those procedures
behave as though the string is first converted to a text, so
passing a text is likely to be more efficient than passing a string.
</p>
<!--========================================================================-->
<h3><a name="PerformanceRequirements">Performance requirements</a></h3>
<p>
A few procedures are required to execute in O(1) time:
<code>text?</code>, <code>textual?</code>,
<code>text-length</code>, and <code>text-ref</code>.
</p>
<p>
If the first two arguments passed to <code>textual-contains</code> and
<code>textual-contains-right</code> are texts, then those procedures
must run in O(<var>m n</var>) time, where <var>m</var>
and <var>n</var> are the lengths of the two subtexts specified by
their arguments.
If either of the first two arguments is a string, there is no such
requirement.
</p>
<p>
The other procedures specified by this SRFI should run in
amortized linear time, not counting time spent in procedures and
predicates that were passed as arguments.
That is not an absolute requirement, but the sample implementations
are designed to deliver that level of performance for most procedures
provided none of their textual arguments are strings.
When strings are passed as arguments, the running time is unlikely
to be linear unless <code>string-ref</code> runs in constant time,
and that is not required by any of the Scheme standards.
</p>
<p>
Indeed, this SRFI was designed to make efficient text processing
easier in systems whose <code>string-ref</code> procedure does not
run in constant time. For efficiency, portable code should use
strings only for fairly short sequences of characters.
Representations with guaranteed efficiency (such as the immutable
texts of this SRFI) should be used for longer texts.
</p>
<p>
<i>Note:</i>
A procedure that runs in O(1) time does not necessarily take the
same time for all inputs.
Furthermore O(1) = O(1000), so procedures that run in O(1) time
can still be quite slow.
The <code>text-ref</code> procedure, for example, may have worst
cases for which it is hundreds of times slower than <code>text?</code>.
Even the average case for <code>text-ref</code> is likely to be
several times as slow as the worst case for <code>text?</code>.
</p>
<!--========================================================================-->
<h3><a name="Unicode">Unicode</a></h3>
<p>
During the early development of Unicode, its designers believed
a 16-bit character set would suffice,
which is why Java's <code>char</code> type has only 16 bits.
When Unicode expanded to 1114112 code points, 16 bits were
no longer enough to encode all Unicode characters.
</p>
<p>
The Unicode standard defines three encoding forms for arbitrary
sequences of Unicode characters:
</p>
<dl>
<dt>
UTF-32
</dt>
<dd>
is a fixed-width encoding in which every character is represented
by a straightforward 32-bit representation of its code point.
</dd>
<dt>
UTF-16
</dt>
<dd>
is a variable-width encoding in which the most common characters
are represented by 16-bit representations of their code
points, but characters outside the Basic Multilingual Plane (BMP)
are represented by a surrogate pair consisting of two consecutive
16-bit code units.
</dd>
<dt>
UTF-8
</dt>
<dd>
is a variable-width encoding in which ASCII characters
are represented by 8-bit representations of their code
points, but other characters are encoded by a sequence of two,
three, or four 8-bit code units.
</dd>
</dl>
<p>
UTF-32 is a convenient internal representation and is used as such
by several string libraries for C, C++, and Python,
but it is the least compact of the three representations
and is seldom used in files.
UTF-16 is convenient for applications that use only the BMP, and
supports fast sequential processing of arbitrary Unicode;
variants of UTF-16 are used by Windows for files and by Java and
C# as an internal representation.
UTF-8 is upwardly compatible with the ASCII encoding and supports
fast sequential processing of arbitrary Unicode;
it is widely used for files on non-Windows machines and is also
used by some C libraries.
</p>
<p>
The Scheme programming language does not expose the internal
representation of strings.
Some implementations of Scheme use UTF-32 or a similar encoding,
which makes <code>string-length</code>, <code>string-ref</code>,
and <code>string-set!</code> run in O(1) time.
Some implementations of Scheme use UTF-16, which saves space
at the expense of making <code>string-ref</code> take
time proportional to the length of a string.
Some implementations of Scheme use UTF-8, which saves even more space
for ASCII strings while making <code>string-ref</code> run in
linear time.
</p>
<p>
Although Scheme's string data type allows portable code to use strings
independently of their internal representation, the variation
in performance between implementations has created a problem
for programs that use long strings.
In some systems, long strings are inefficient with respect to space;
in other systems, long strings are inefficient with respect to time.
</p>
<p>
The portable solution to this dilemma is to use Scheme's mutable
strings only for buffers and other relatively short sequences of
characters, while using the immutable texts defined by this SRFI
for long sequences of characters.
</p>
<p>
<i>Note:</i>
SRFI 130 suggests an alternative solution: Portable code should
process strings sequentially using cursors instead of indexes,
and should avoid mutation of strings by using vectors of characters
instead,
while hoping all major implementations of Scheme will soon convert
their strings to use compact internal representations such as UTF-8
or UTF-16.
That hope is unlikely to be realized, because a lot of legacy code
assumes <code>string-ref</code> runs in O(1) time, as recommended
by the R6RS, and mutable strings represented in UTF-32 or similar
are more efficient than vectors of characters with respect to both
time and space.
At present, several implementations of Scheme support Unicode while
providing <code>string-ref</code> and <code>string-set!</code> procedures
that run in O(1) time; making those operations run asymptotically
slower would displease some users of those systems.
</p>
<!--========================================================================-->
<h2><a name="Notation">Notation</a></h2>
<p>
In the following procedure specifications:
<ul>
<li>A <var>text</var> argument is an immutable text.</li>
<li>A <var>textual</var> argument is an immutable text or a string.</li>
<li>A <var>char</var> argument is a character.</li>
<li>An <var>idx</var> argument is an exact non-negative integer
specifying a valid character index into a text or string.
The valid character indexes of a text or string <var>textual</var>
of length <var>n</var> are the exact integers <var>idx</var> satisfying
0 <= <var>idx</var> < <var>n</var>.
</li>
<li>A <var>k</var> argument or result is a <em>position</em>:
an exact non-negative
integer that is either a valid character index for one of the
textual arguments or is the length of a textual argument.
</li>
<li><var>start</var> and <var>end</var> arguments are positions
specifying
a half-open interval of indexes for a subtext or substring.
When omitted, <var>start</var> defaults to 0 and <var>end</var>
to the length of the corresponding <var>textual</var> argument.
It is an error unless
0 <= <var>start</var> <= <var>end</var>
<= <code>(textual-length <var>textual</var>)</code>;
the sample implementations detect that error and raise an exception.
</li>
<li>A <var>len</var> or <var>nchars</var> argument is an exact
non-negative integer specifying some number of characters,
usually the length of a text or string.</li>
<li>A <var>pred</var> argument is a unary character predicate,
taking a character as its one argument and returning a value
that will be interpreted as true or false.
Unless noted otherwise, as with <code>textual-every</code> and
<code>textual-any</code>,
all predicates passed to procedures specified in this SRFI may be
called in any order and any number of times.
It is an error if <var>pred</var> has side effects or
does not behave functionally (returning the same result whenever
it is called with the same character);
the sample implementations do not detect those errors.
</li>
<li>An <var>obj</var> argument may be any value at all.</li>
</ul>
<p class=continue>
It is an error to pass values that violate the specification above.
</p>
<p>
Arguments given in square brackets are optional. Unless otherwise noted in the
text describing the procedure, any prefix of these optional arguments may
be supplied, from zero arguments to the full list. When a procedure returns
multiple values, this is shown by listing the return values in square
brackets, as well. So, for example, the procedure with signature
<pre class="code-example">
halts? <var>f [x init-store]</var> → <var>[boolean integer]</var>
</pre>
<p>
would take one (<var>f</var>), two (<var>f</var>, <var>x</var>)
or three (<var>f</var>, <var>x</var>, <var>init-store</var>) input arguments,
and return two values, a boolean and an integer.
</p>
<p>
An argument followed by "<code>...</code>" means zero or more elements.