Shlex is a PHP extension written in C. This extension implements the functionality of the shlex library in Python. In order to make users more familiar with the Shlex extension, the class implemented by the extension is basically the same as the python shlex library in terms of property and method names. The interface documentation is also modified from the python shlex library interface documentation.
The Shlex makes it easy to write lexical analyzers for simple syntaxes resembling that of the Unix shell. This will often be useful for writing minilanguages or for parsing quoted strings.
- Requirement
- Installation
- Functions
- Classes and methods
- Shlex
- Properties
- instream
- infile
- posix
- eof
- commenters
- wordchars
- whitespace
- whitespaceSplit
- quotes
- escape
- escapedquotes
- state
- pushback
- lineno
- debug
- token
- filestack
- source
- punctuationChars
- _punctuationChars
- Methods
- Shlex::__construct
- Shlex::__destruct
- Shlex::key
- Shlex::next
- Shlex::rewind
- Shlex::current
- Shlex::valid
- Shlex::pushToken
- Shlex::pushSource
- Shlex::popSource
- Shlex::getToken
- Shlex::readToken
- Shlex::sourcehook
- Shlex::errorLeader
- ShlexException
- Shlex
- PHP 7.0 +
- Linux/OSX
phpize
./configure
make && make install
Windows system is currently not supported.
Split the string s using shell-like syntax.
array shlex_split( string|resource|null $s [, bool $comments = false [, bool $posix = true ]] )
Split the string s using shell-like syntax.
Note:
Since the shlex_split() function instantiates a shlex instance, passing null for s will read the string to split from standard input.
If comments is false (the default), the parsing of comments in the given string will be disabled (setting the commenters attribute of the shlex instance to the empty string).
This function operates in POSIX mode by default, but uses non-POSIX mode if the posix argument is false.
Returns an array of split strings.
<?php
$s = "foo#bar";
$ret = shlex_split($s, true);
var_dump($ret);
?>
The above example will output:
array(1) {
[0] =>
string(3) "foo"
}
Return a shell-escaped version of the string s.
string shlex_quote( string $s )
The string to be escaped.
The returned value is a string that can safely be used as one token in a shell command line, for cases where you cannot use a list.
<?php
// If the output is executed, it will cause the index.php file to be deleted.
$filename = "somefile; rm -rf index.php";
$command = sprintf("ls -l %s", $filename);
echo $command;
echo "\n";
// shlex_quote() blocked the vulnerability
$command = sprintf("ls -l %s", shlex_quote($filename));
echo $command;
echo "\n";
// remote connection
$remoteCommand = sprintf("ssh home %s", shlex_quote($command));
echo $remoteCommand;
echo "\n";
?>
The above example will output:
ls -l somefile; rm -rf index.php
ls -l 'somefile; rm -rf index.php'
ssh home 'ls -l '"'"'somefile; rm -rf index.php'"'"''
A Shlex instance or subclass instance is a lexical analyzer object.
Shlex implements Iterator {
/* Properties */
public resource|null $instream = null;
public string|null $infile = null;
private bool|null $posix = null;
public string|null $eof = null;
public string $commenters = '#';
public string $wordchars = 'abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_';
public string $whitespace = " \t\r\n";
public bool $whitespaceSplit = false;
public string $quotes = '\'"';
public string $escape = '\\';
public string $escapedquotes = '"';
private string $state = ' ';
private array $pushback = [];
public int $lineno = 1;
public int $debug = 0;
public string $token = '';
private array $filestack = [];
public string|null $source = null;
public string|null $punctuationChars = null;
private array|null $_punctuationChars = null;
/* Methods */
public void function __construct( [ string|resource|null $instream = null [, string|null $infile = null [, bool $posix = false [, string|bool|null $punctuationChars = false ]]]]);
public void function __destruct( void );
public void function key( void );
public void function next( void );
public void function rewind( void );
public string|null function current( void );
public bool function valid( void );
public void function pushToken( string $tok );
public void function pushSource( string|resource $newstream, string|null $newfile = null );
public void function popSource( void );
public string|null|ShlexException function getToken( void );
public string|null|ShlexException function readToken( void );
public array function sourcehook( string $newfile );
public string function errorLeader( string $infile = null, int|null $lineno = null );
}
The input stream from which this Shlex instance is reading characters.
The name of the current input file, as initially set at class instantiation time or stacked by later source requests. It may be useful to examine this when constructing error messages.
Token used to determine end of file. This will be set to the empty string (''), in non-POSIX mode, and to null in POSIX mode.
The string of characters that are recognized as comment beginners. All characters from the comment beginner to end of line are ignored. Includes just '#' by default.
The string of characters that will accumulate into multi-character tokens. By default, includes all ASCII alphanumerics and underscore. In POSIX mode, the accented characters in the Latin-1 set are also included. If punctuationChars is not empty, the characters ~-./*?=, which can appear in filename specifications and command line parameters, will also be included in this attribute, and any characters which appear in punctuationChars will be removed from wordchars if they are present there.
Characters that will be considered whitespace and skipped. Whitespace bounds tokens. By default, includes space, tab, linefeed and carriage-return.
If true, tokens will only be split in whitespaces. This is useful, for example, for parsing command lines with Shlex, getting tokens in a similar way to shell arguments. If this attribute is true, punctuationChars will have no effect, and splitting will happen only on whitespaces. When using punctuationChars, which is intended to provide parsing closer to that implemented by shells, it is advisable to leave whitespaceSplit as false (the default value).
Characters that will be considered string quotes. The token accumulates until the same quote is encountered again (thus, different quote types protect each other as in the shell.) By default, includes ASCII single and double quotes.
Characters that will be considered as escape. This will be only used in POSIX mode, and includes just '' by default.
Characters in quotes that will interpret escape characters defined in escape. This is only used in POSIX mode, and includes just '"' by default.
Source line number (count of newlines seen so far plus one).
If this attribute is numeric and 1 or more, a Shlex instance will print verbose progress output on its behavior. If you need to use this, you can read the module source code to learn the details.
The token buffer. It may be useful to examine this when catching exceptions.
This attribute is null by default. If you assign a string to it, that string will be recognized as a lexical-level inclusion request similar to the source keyword in various shells. That is, the immediately following token will be opened as a filename and input will be taken from that stream until EOF, at which point the fclose() method of that stream will be called and the input source will again become the original input stream. Source requests may be stacked any number of levels deep.
Characters that will be considered punctuation. Runs of punctuation characters will be returned as a single token. However, note that no semantic validity checking will be performed: for example, ‘>>>’ could be returned as a token, even though it may not be recognised as such by shells.
Constructor
public void function Shlex::__construct( [ string|resource|null $instream = null [, string|null $infile = null [, bool $posix = false [, string|bool|null $punctuationChars = false ]]]])
The instream argument, if present, specifies where to read characters from. It must be a resource type variable (can be read by fread( )), or a string. If no argument is given, input will be taken from php://stdin.
The second optional argument is a filename string, which sets the initial value of the infile attribute. If the instream argument is null, then this infile argument is always null.
The posix argument defines the operational mode: when posix is false (default), the Shlex instance will operate in compatibility mode. When operating in POSIX mode, Shlex will try to be as close as possible to the POSIX shell parsing rules.
The punctuationChars argument provides a way to make the behaviour even closer to how real shells parse. This can take a number of values: the default value, false. If set to true, then parsing of the characters ();<>|& is changed: any run of these characters (considered punctuation characters) is returned as a single token. If set to a non-empty string of characters, those characters will be used as the punctuation characters. Any characters in the wordchars attribute that appear in punctuationChars will be removed from wordchars.
No value is returned.
<?php
$instance = new Shlex("a && b || c", null, false, "|");
$list = [];
foreach ($instance as $value) {
$list[] = $value;
}
var_dump($list);
?>
The above example will output:
array(6) {
[0] =>
string(1) "a"
[1] =>
string(1) "&"
[2] =>
string(1) "&"
[3] =>
string(1) "b"
[4] =>
string(2) "||"
[5] =>
string(1) "c"
}
Destructor
public void function Shlex::__destruct( void )
Used to release resource objects held by Shlex objects. Internally, fclose( ) is called to close the file handle.
No parameters.
No value is returned.
No examples.
There is no practical use for the key method of the Iterator interface.
public void function Shlex::key( void )
No parameters.
No value is returned.
No examples.
There is no practical use for the next method of the Iterator interface.
public void function Shlex::next( void )
No parameters.
No value is returned.
No examples.
There is no practical use for the rewind method of the Iterator interface.
public void function Shlex::rewind( void )
No parameters.
No value is returned.
No examples.
Returns the token value read by Shlex this iteration.
public string|null function Shlex::current( void )
No parameters.
Returns the token value read by Shlex this iteration.
No examples.
Determine if this iteration is valid.
public bool function Shlex::valid( void )
No parameters.
Valid if true is returned, false is invalid.
Note:
Due to the implementation of this class, iteratively reading the next element is also called inside the method. So the next() method is invalid.
No examples.
Push the argument onto the token stack.
public void function Shlex::pushToken( string $tok )
The parameter being pushed.
No value is returned.
No examples.
Push an input source stream onto the input stack.
public void function Shlex::pushSource( string|resource $newstream, string|null $newfile = null );
The input source stream being pushed.
If the filename argument is specified it will later be available for use in error messages. This is the same method used internally by the sourcehook() method.
No value is returned.
No examples.
Pop the last-pushed input source from the input stack. This is the same method used internally when the lexer reaches EOF on a stacked input stream.
public void function Shlex::popSource( void )
No parameters.
No value is returned.
No examples.
Return a token.
public string|null|ShlexException function Shlex::getToken( void )
No parameters.
If tokens have been stacked using pushToken(), pop a token off the stack. Otherwise, read one from the input stream. If reading encounters an immediate end-of-file, eof is returned (the empty string ('') in non-POSIX mode, and null in POSIX mode).
No examples.
Read a raw token.
public string|null|ShlexException function Shlex::readToken( void )
Read a raw token. Ignore the pushback stack, and do not interpret source requests. (This is not ordinarily a useful entry point, and is documented here only for the sake of completeness.)
No parameters.
Return a raw token.
No examples.
public array function Shlex::sourcehook( string $newfile )
When Shlex detects a source request (see source below) this method is given the following token as argument, and expected to return a array of a filename and an open file-like object.
Normally, this method first strips any quotes off the argument. If the result is an absolute pathname, or there was no previous source request in effect, or the previous source was a stream (such as php://stdin), the result is left alone. Otherwise, if the result is a relative pathname, the directory part of the name of the file immediately before it on the source inclusion stack is prepended (this behavior is like the way the C preprocessor handles #include "file.h").
The result of the manipulations is treated as a filename, and returned as the first component of the tuple, with fopen() called on it to yield the second component. (Note: this is the reverse of the order of arguments in instance initialization!)
This hook is exposed so that you can use it to implement directory search paths, addition of file extensions, and other namespace hacks. There is no corresponding ‘close’ hook, but a shlex instance will call the fclose() method of the sourced input stream when it returns EOF.
For more explicit control of source stacking, use the pushSource() and popSource() methods.
file path.
Return a array of a filename and an open file-like object.
No examples.
Return an error message leader in the format of a Unix C compiler error label.
public string function Shlex::errorLeader( string $infile = null, int|null $lineno = null )
This method generates an error message leader in the format of a Unix C compiler error label; the format is '"%s", line %d: ', where the %s is replaced with the name of the current source file and the %d with the current input line number (the optional arguments can be used to override these).
This convenience is provided to encourage Shlex users to generate error messages in the standard, parseable format understood by Emacs and other Unix tools.
The name of the current source file.
The current input line number.
Return an error message leader in the format of a Unix C compiler error label.
No examples.
Shlex's exception class
This class is primarily used for exceptions thrown when the Shlex class internally performs an error.
ShlexException extends Exception {}
<?php
throw new ShlexException('No escaped character');
?>