1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387
|
NAME
Devel::Declare - Adding keywords to perl, in perl
SYNOPSIS
use Method::Signatures;
# or ...
use MooseX::Declare;
# etc.
# Use some new and exciting syntax like:
method hello (Str :$who, Int :$age where { $_ > 0 }) {
$self->say("Hello ${who}, I am ${age} years old!");
}
DESCRIPTION
Devel::Declare can install subroutines called declarators which locally
take over Perl's parser, allowing the creation of new syntax.
This document describes how to create a simple declarator.
USAGE
We'll demonstrate the usage of "Devel::Declare" with a motivating
example: a new "method" keyword, which acts like the builtin "sub", but
automatically unpacks $self and the other arguments.
package My::Methods;
use Devel::Declare;
Creating a declarator with "setup_for"
You will typically create
sub import {
my $class = shift;
my $caller = caller;
Devel::Declare->setup_for(
$caller,
{ method => { const => \&parser } }
);
no strict 'refs';
*{$caller.'::method'} = sub (&) {};
}
Starting from the end of this import routine, you'll see that we're
creating a subroutine called "method" in the caller's namespace. Yes,
that's just a normal subroutine, and it does nothing at all (yet!) Note
the prototype "(&)" which means that the caller would call it like so:
method {
my ($self, $arg1, $arg2) = @_;
...
}
However we want to be able to call it like this
method foo ($arg1, $arg2) {
...
}
That's why we call "setup_for" above, to register the declarator
'method' with a custom parser, as per the next section. It acts on an
optype, usually 'const' as above. (Other valid values are 'check' and
'rv2cv').
For a simpler way to install new methods, see also
Devel::Declare::MethodInstaller::Simple
Writing a parser subroutine
This subroutine is called at *compilation* time, and allows you to read
the custom syntaxes that we want (in a syntax that may or may not be
valid core Perl 5) and munge it so that the result will be parsed by the
"perl" compiler.
For this example, we're defining some globals for convenience:
our ($Declarator, $Offset);
Then we define a parser subroutine to handle our declarator. We'll look
at this in a few chunks.
sub parser {
local ($Declarator, $Offset) = @_;
"Devel::Declare" provides some very low level utility methods to parse
character strings. We'll define some useful higher level routines below
for convenience, and we can use these to parse the various elements in
our new syntax.
Notice how our parser subroutine is invoked at compile time, when the
"perl" parser is pointed just *before* the declarator name.
skip_declarator; # step past 'method'
my $name = strip_name; # strip out the name 'foo', if present
my $proto = strip_proto; # strip out the prototype '($arg1, $arg2)', if present
Now we can prepare some code to 'inject' into the new subroutine. For
example we might want the method as above to have "my ($self, $arg1,
$arg2) = @_" injected at the beginning of it. We also do some clever
stuff with scopes that we'll look at shortly.
my $inject = make_proto_unwrap($proto);
if (defined $name) {
$inject = scope_injector_call().$inject;
}
inject_if_block($inject);
We've now managed to change "method ($arg1, $arg2) { ... }" into "method
{ injected_code; ... }". This will compile... but we've lost the name of
the method!
In a cute (or horrifying, depending on your perspective) trick, we
temporarily change the definition of the subroutine "method" itself, to
specialise it with the $name we stripped, so that it assigns the code
block to that name.
Even though the *next* time "method" is compiled, it will be redefined
again, "perl" caches these definitions in its parse tree, so we'll
always get the right one!
Note that we also handle the case where there was no name, allowing an
anonymous method analogous to an anonymous subroutine.
if (defined $name) {
$name = join('::', Devel::Declare::get_curstash_name(), $name)
unless ($name =~ /::/);
shadow(sub (&) { no strict 'refs'; *{$name} = shift; });
} else {
shadow(sub (&) { shift });
}
}
Parser utilities in detail
For simplicity, we're using global variables like $Offset in these
examples. You may prefer to look at Devel::Declare::Context::Simple,
which encapsulates the context much more cleanly.
"skip_declarator"
This simple parser just moves across a 'token'. The common case is to
skip the declarator, i.e. to move to the end of the string 'method' and
before the prototype and code block.
sub skip_declarator {
$Offset += Devel::Declare::toke_move_past_token($Offset);
}
"toke_move_past_token"
This builtin parser simply moves past a 'token' (matching
"/[a-zA-Z_]\w*/") It takes an offset into the source document, and skips
past the token. It returns the number of characters skipped.
"strip_name"
This parser skips any whitespace, then scans the next word (again
matching a 'token'). We can then analyse the current line, and
manipulate it (using pure Perl). In this case we take the name of the
method out, and return it.
sub strip_name {
skipspace;
if (my $len = Devel::Declare::toke_scan_word($Offset, 1)) {
my $linestr = Devel::Declare::get_linestr();
my $name = substr($linestr, $Offset, $len);
substr($linestr, $Offset, $len) = '';
Devel::Declare::set_linestr($linestr);
return $name;
}
return;
}
"toke_scan_word"
This builtin parser, given an offset into the source document, matches a
'token' as above but does not skip. It returns the length of the token
matched, if any.
"get_linestr"
This builtin returns the full text of the current line of the source
document.
"set_linestr"
This builtin sets the full text of the current line of the source
document. Beware that injecting a newline into the middle of the line is
likely to fail in surprising ways. Generally, Perl's parser can rely on
the `current line' actually being only a single line. Use other kinds of
whitespace instead, in the code that you inject.
"skipspace"
This parser skips whitsepace.
sub skipspace {
$Offset += Devel::Declare::toke_skipspace($Offset);
}
"toke_skipspace"
This builtin parser, given an offset into the source document, skips
over any whitespace, and returns the number of characters skipped.
"strip_proto"
This is a more complex parser that checks if it's found something that
starts with '(' and returns everything till the matching ')'.
sub strip_proto {
skipspace;
my $linestr = Devel::Declare::get_linestr();
if (substr($linestr, $Offset, 1) eq '(') {
my $length = Devel::Declare::toke_scan_str($Offset);
my $proto = Devel::Declare::get_lex_stuff();
Devel::Declare::clear_lex_stuff();
$linestr = Devel::Declare::get_linestr();
substr($linestr, $Offset, $length) = '';
Devel::Declare::set_linestr($linestr);
return $proto;
}
return;
}
"toke_scan_str"
This builtin parser uses Perl's own parsing routines to match a
"stringlike" expression. Handily, this includes bracketed expressions
(just think about things like "q(this is a quote)").
Also it Does The Right Thing with nested delimiters (like "q(this (is
(a) quote))").
It returns the effective length of the expression matched. Really, what
it returns is the difference in position between where the string
started, within the buffer, and where it finished. If the string
extended across multiple lines then the contents of the buffer may have
been completely replaced by the new lines, so this position difference
is not the same thing as the actual length of the expression matched.
However, because moving backward in the buffer causes problems, the
function arranges for the effective length to always be positive,
padding the start of the buffer if necessary.
Use "get_lex_stuff" to get the actual matched text, the content of the
string. Because of the behaviour around multiline strings, you can't
reliably get this from the buffer. In fact, after the function returns,
you can't rely on any content of the buffer preceding the end of the
string.
If the string being scanned is not well formed (has no closing
delimiter), "toke_scan_str" returns "undef". In this case you cannot
rely on the contents of the buffer.
"get_lex_stuff"
This builtin returns what was matched by "toke_scan_str". To avoid
segfaults, you should call "clear_lex_stuff" immediately afterwards.
Munging the subroutine
Let's look at what we need to do in detail.
"make_proto_unwrap"
We may have defined our method in different ways, which will result in a
different value for our prototype, as parsed above. For example:
method foo { # undefined
method foo () { # ''
method foo ($arg1) { # '$arg1'
We deal with them as follows, and return the appropriate "my ($self,
...) = @_;" string.
sub make_proto_unwrap {
my ($proto) = @_;
my $inject = 'my ($self';
if (defined $proto) {
$inject .= ", $proto" if length($proto);
$inject .= ') = @_; ';
} else {
$inject .= ') = shift;';
}
return $inject;
}
"inject_if_block"
Now we need to inject it after the opening '{' of the method body. We
can do this with the building blocks we defined above like "skipspace"
and "get_linestr".
sub inject_if_block {
my $inject = shift;
skipspace;
my $linestr = Devel::Declare::get_linestr;
if (substr($linestr, $Offset, 1) eq '{') {
substr($linestr, $Offset+1, 0) = $inject;
Devel::Declare::set_linestr($linestr);
}
}
"scope_injector_call"
We want to be able to handle both named and anonymous methods. i.e.
method foo () { ... }
my $meth = method () { ... };
These will then get rewritten as
method { ... }
my $meth = method { ... };
where 'method' is a subroutine that takes a code block. Spot the
problem? The first one doesn't have a semicolon at the end of it! Unlike
'sub' which is a builtin, this is just a normal statement, so we need to
terminate it. Luckily, using "B::Hooks::EndOfScope", we can do this!
use B::Hooks::EndOfScope;
We'll add this to what gets 'injected' at the beginning of the method
source.
sub scope_injector_call {
return ' BEGIN { MethodHandlers::inject_scope }; ';
}
So at the beginning of every method, we are passing a callback that will
get invoked at the *end* of the method's compilation... i.e. exactly
then the closing '}' is compiled.
sub inject_scope {
on_scope_end {
my $linestr = Devel::Declare::get_linestr;
my $offset = Devel::Declare::get_linestr_offset;
substr($linestr, $offset, 0) = ';';
Devel::Declare::set_linestr($linestr);
};
}
Shadowing each method.
"shadow"
We override the current definition of 'method' using "shadow".
sub shadow {
my $pack = Devel::Declare::get_curstash_name;
Devel::Declare::shadow_sub("${pack}::${Declarator}", $_[0]);
}
For a named method we invoked like this:
shadow(sub (&) { no strict 'refs'; *{$name} = shift; });
So in the case of a "method foo { ... }", this call would redefine
"method" to be a subroutine that exports 'sub foo' as the (munged)
contents of "{...}".
The case of an anonymous method is also cute:
shadow(sub (&) { shift });
This means that
my $meth = method () { ... };
is rewritten with "method" taking the codeblock, and returning it as is
to become the value of $meth.
"get_curstash_name"
This returns the package name *currently being compiled*.
"shadow_sub"
Handles the details of redefining the subroutine.
SEE ALSO
One of the best ways to learn "Devel::Declare" is still to look at
modules that use it:
<http://cpants.perl.org/dist/used_by/Devel-Declare>.
AUTHORS
Matt S Trout - <mst@shadowcat.co.uk> - original author
Company: http://www.shadowcat.co.uk/ Blog: http://chainsawblues.vox.com/
Florian Ragwitz <rafl@debian.org> - maintainer
osfameron <osfameron@cpan.org> - first draft of documentation
COPYRIGHT AND LICENSE
This library is free software under the same terms as perl itself
Copyright (c) 2007, 2008, 2009 Matt S Trout
Copyright (c) 2008, 2009 Florian Ragwitz
stolen_chunk_of_toke.c based on toke.c from the perl core, which is
Copyright (C) 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
2000, 2001, 2002, 2003, 2004, 2005, 2006, by Larry Wall and others
|