1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage Express 2.0">
<title>Boost Offset Separator</title>
<!--
-- Copyright John Bandela 2001
--
-- Permission to use, copy, modify, distribute and sell this software
-- and its documentation for any purpose is hereby granted without fee,
-- provided that the above copyright notice appears in all copies and
-- that both that copyright notice and this permission notice appear
-- in supporting documentation. Jeremy Siek makes no
-- representations about the suitability of this software for any
-- purpose. It is provided "as is" without express or implied warranty.
-->
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#0000EE"
vlink="#551A8B" alink="#FF0000">
<p><img src="../../c++boost.gif" alt="C++ Boost" width="277"
height="86"> <br>
</p>
<h1 align="center">Offset Separator</h1>
<pre>
class offset_separator
</pre>
<p>The <tt>offset_separator</tt> class is an implementation of
the <a href="tokenizerfunction.htm">TokenizerFunction</a> concept
that can be used with the <a href="tokenizer.htm">tokenizer</a>
class to break text up into tokens. The <tt>offset_separator</tt>
breaks a sequence of <tt>Char</tt>'s into strings based on a
sequence of offsets. For example, if you had the string "12252001"
and offsets (2,2,4) it would break the string into 12 25 2001.
Here is an example.</p>
<h2>Example</h2>
<pre>// simple_example_3.cpp
#include<iostream>
#include<boost/tokenizer.hpp>
#include<string>
int main(){
using namespace std;
using namespace boost;
string s = "12252001";
int offsets[] = {2,2,4};
offset_separator f(offsets, offsets+3);
tokenizer<offset_separator> tok(s,f);
for(tokenizer<offset_separator>::iterator beg=tok.begin(); beg!=tok.end();++beg){
cout << *beg << "\n";
}
}</pre>
<p> </p>
<h2>Construction and Usage</h2>
<p>The offset_separator has 1 constructor of interest. (The
default constructor is just there to make some compilers happy).
The declaration is below</p>
<pre>template<typename Iter>
offset_separator(Iter begin,Iter end,bool bwrapoffsets = true, bool breturnpartiallast = true)
</pre>
<table border="1">
<tr>
<td><p align="center"><strong>Parameter</strong></p>
</td>
<td><p align="center"><strong>Description</strong></p>
</td>
</tr>
<tr>
<td>begin, end</td>
<td>Specify the sequence of integer offsets.</td>
</tr>
<tr>
<td>bwrapoffsets</td>
<td>Tells whether to wrap around to the beginning of the
offsets when the all the offsets have been used. For
example the string "1225200101012002" with
offsets (2,2,4) with bwrapoffsets to true, would parse to
12 25 2001 01 01 2002. With bwrapoffsets to false, it
would parse to 12 25 2001 and then stop because all the
offsets have been used.</td>
</tr>
<tr>
<td>breturnpartiallast</td>
<td>Tells whether, when the parsed sequence terminates
before yielding the number of characters in the current
offset, to create a token with what was parsed, or to
ignore it. For example the string "122501" with
offsets (2,2,4) with breturnpartiallast set to true will
parse to 12 25 01. With it set to false, it will parse to
12 25 and then will stop because there are only 2
characters left in the sequence instead of the 4 that
should have been there.</td>
</tr>
</table>
<p>To use this class, pass an object of it anywhere a
TokenizerFunction is required. If you default constructruct the
object, it will just return every character in the parsed
sequence as a token. (ie it defaults to an offset of 1, and
bwrapoffsets is true).</p>
<p> </p>
<h2>Model of</h2>
<p><a href="tokenizerfunction.htm">TokenizerFunction</a> </p>
<hr>
<p> Copyright John R. Bandela 2001. Permission to copy, use,
modify, sell and distribute this document is granted provided
this copyright notice appears in all copies. This document is
provided "as is" without express or implied warranty,
and with no claim as to its suitability for any purpose.</p>
</body>
</html>
|
