summaryrefslogtreecommitdiffstats
path: root/bin/sh/expand.c
Commit message (Collapse)AuthorAgeFilesLines
* sh: Make various functions static.jilles2012-01-011-1/+2
|
* sh: Make patmatch() non-recursive.jilles2012-01-011-37/+58
|
* sh: Use dirent.d_type in pathname generation.jilles2011-12-281-0/+4
| | | | | | This improves performance for globs where a slash or another component follows a component with metacharacters by eliminating unnecessary attempts to open directories that are not.
* sh: Cache de->d_namlen in a local variable.jilles2011-12-281-6/+8
|
* sh: Add support for named character classes in bracket expressions.jilles2011-06-151-1/+37
| | | | | Example: case x in [[:alpha:]]) echo yes ;; esac
* sh: Fix duplicate prototypes for builtins.jilles2011-06-131-0/+1
| | | | | | Have mkbuiltins write the prototypes for the *cmd functions to builtins.h instead of builtins.c and include builtins.h in more .c files instead of duplicating prototypes for *cmd functions in other headers.
* sh: Save/restore changed variables in optimized command substitution.jilles2011-06-121-72/+0
| | | | | | | | | | | In optimized command substitution, save and restore any variables changed by expansions (${var=value} and $((var=assigned))), instead of trying to determine if an expansion may cause such changes. If $! is referenced in optimized command substitution, do not cause jobs to be remembered longer. This fixes $(jobs $!) again, simplifies the man page and shortens the code.
* sh: Fix locale-dependent ranges in bracket expressions.jilles2011-06-121-4/+4
| | | | | | | | When I added UTF-8 support in r221646, the LC_COLLATE-based ordering broke because of sign extension of char. Because of libc restrictions, this does not work for UTF-8. For UTF-8 locales, ranges always use character code order.
* sh: Do parameter expansion before printing PS4 (set -x).jilles2011-06-091-0/+1
| | | | | | | | | | | | | | | | The function name expandstr() and the general idea of doing this kind of expansion by treating the text as a here document without end marker is from dash. All variants of parameter expansion and arithmetic expansion also work (the latter is not required by POSIX but it does not take extra code and many other shells also allow it). Command substitution is prevented because I think it causes too much code to be re-entered (for example creating an unbounded recursion of trace lines). Unfortunately, our LINENO is somewhat crude, otherwise PS4='$LINENO+ ' would be quite useful.
* sh: Fix unquoted $@/$* if IFS=''.jilles2011-05-271-2/+5
| | | | | | If IFS is null, unquoted $@/$* should still expand to separate words. This differs from quoted $@ (which does not depend on IFS) in that pathname generation is performed and empty words are removed.
* sh: Add UTF-8 support to pattern matching.jilles2011-05-081-12/+50
| | | | | | | | | | ?, [...] patterns match codepoints instead of bytes. They do not match invalid sequences. [...] patterns must not contain invalid sequences otherwise they will not match anything. This is so that ${var#?} removes the first codepoint, not the first byte, without putting UTF-8 knowledge into the ${var#pattern} code. However, * continues to match any string and an invalid sequence matches an identical invalid sequence. (This differs from fnmatch(3).)
* sh: Add UTF-8 support to ${#var}.jilles2011-05-071-3/+13
| | | | | If the current locale uses UTF-8, ${#var} counts codepoints (more precisely, bytes b with (b & 0xc0) != 0x80).
* Fix typos - remove duplicate "the".brucec2011-02-211-1/+1
| | | | | | PR: bin/154928 Submitted by: Eitan Adler <lists at eitanadler.com> MFC after: 3 days
* sh: Remove comment mentioning herefd, which is gone.jilles2011-02-021-2/+1
|
* sh: Don't do optimized command substitution if expansions have side effects.jilles2010-12-281-0/+72
| | | | | | | | | | | | | | | | | | | | | | | | | Before considering to execute a command substitution in the same process, check if any of the expansions may have a side effect; if so, execute it in a new process just like happens if it is not a single simple command. Although the check happens at run time, it is a static check that does not depend on current state. It is triggered by: - expanding $! (which may cause the job to be remembered) - ${var=value} default value assignment - assignment operators in arithmetic - parameter substitutions in arithmetic except ${#param}, $$, $# and $? - command substitutions in arithmetic This means that $((v+1)) does not prevent optimized command substitution, whereas $(($v+1)) does, because $v might expand to something containing assignment operators. Scripts should not depend on these exact details for correctness. It is also imaginable to have the shell fork if and when a side effect is encountered or to create a new temporary namespace for variables. Due to the $! change, the construct $(jobs $!) no longer works. The value of $! should be stored in a variable outside command substitution first.
* sh: Allow arbitrary large numbers in CHECKSTRSPACE.jilles2010-12-261-3/+4
| | | | | | Reduce "stack string" API somewhat and simplify code. Add a check for integer overflow of the "stack string" length (probably incomplete).
* Remove dead code.uqs2010-12-181-2/+0
| | | | | | | | c is assigned 0 and *loc is pointing to NULL, so c!=0 cannot be true, and dereferencing loc would be a bad idea anyway. Coverity Prevent: CID 5113 Reviewed by: jilles
* sh: Fix corruption of command substitutions with special chars after newlinejilles2010-12-161-2/+2
| | | | | | | | The CTLESC byte to protect a special character was output before instead of after a newline directly preceding the special character. The special handling of newlines is because command substitutions discard all trailing newlines.
* sh: Remove the herefd hack.jilles2010-12-121-8/+0
| | | | | | | | | | | | | | | | | | The herefd hack wrote out partial here documents while expanding them. It seems unnecessary complication given that other expansions just allocate memory. It causes bugs because the stack is also used for intermediate results such as arithmetic expressions. Such places should disable herefd for the duration but not all of them do, and I prefer removing the need for disabling herefd to disabling it everywhere needed. Here documents larger than 1024 bytes will use a bit more CPU time and memory. Additionally this allows a later change to expand here documents in the current shell environment. (This is faster for small here documents but also changes behaviour.) Obtained from: dash
* sh: Replace some macros and repeated code in expand.c with functions.jilles2010-12-111-31/+32
| | | | | No functional change is intended, but the binary is about 1K smaller on i386.
* sh: Code size optimizations to "stack string" memory allocation:jilles2010-11-231-9/+8
| | | | | | | | * Prefer one CHECKSTRSPACE with multiple USTPUTC to multiple STPUTC. * Add STPUTS macro (based on function) and use it instead of loops that add nul-terminated strings to the stack string. No functional change is intended, but code size is about 1K less on i386.
* sh: Code size optimizations to buffered output.jilles2010-11-201-4/+2
| | | | | | This is mainly less use of the outc macro. No functional change is intended, but code size is about 2K less on i386.
* sh: Fix some issues with CTL* bytes and ${var#pat}.jilles2010-10-291-14/+13
| | | | | | | | | | | | | | subevalvar() incorrectly assumed that CTLESC bytes were present iff the expansion was quoted. However, they are present iff various processing such as word splitting is to be done later on. Example: v=@$e@$e@$e@ y="${v##*"$e"}" echo "$y" failed if $e contained the magic CTLESC byte. Exp-run done by: pav (with some other sh(1) changes)
* sh: Do IFS splitting on word in ${v+word} and ${v-word}.jilles2010-10-291-10/+27
| | | | | | | | | | | | | | | | | The code is inspired by NetBSD sh somewhat, but different because we preserve the old Almquist/Bourne/Korn ability to have an unquoted part in a quoted ${v+word}. For example, "${v-"*"}" expands to $v as a single field if v is set, but generates filenames otherwise. Note that this is the only place where we split text literally from the script (the similar ${v=word} assigns to v and then expands $v). The parser must now add additional markers to allow the expansion code to know whether arbitrary characters in substitutions are quoted. Example: for i in ${$+a b c}; do echo $i; done Exp-run done by: pav (with some other sh(1) changes)
* In the spirit of r90111, depend on c89 and remove the "STATIC" macroobrien2010-10-131-35/+35
| | | | and its usage.
* Make DEBUG traces 64-bit clean:jhb2010-10-131-7/+8
| | | | | | | | | - Use %t to print ptrdiff_t values. - Cast a ptrdiff_t value explicitly to int for a field width specifier. While here, sort includes. Submitted by: Garrett Cooper
* Consistently use "STATIC" for all functions in order to be able to setobrien2010-10-131-6/+6
| | | | | | breakpoints with in a debugger. And use naked "static" for variables. Noticed by: bde
* sh: Improve comments in expand.c.jilles2010-09-051-21/+37
|
* sh: Remove remnants of '!!' to negate pattern.jilles2010-08-221-17/+2
| | | | | | | | This Almquist extension was disabled long ago. In pathname generation, components starting with '!!' were treated as containing wildcards, causing unnecessary readdir (which could fail, causing pathname generation to fail while it should not).
* sh: Fix heap-based buffer overflow in pathname generation.jilles2010-08-101-16/+15
| | | | | | | | | | | | | | The buffer for generated pathnames could be too small in some cases. It happened to be always at least PATH_MAX long, so there was never an overflow if the resulting pathnames would be usable. This bug may be abused if a script subjects input from an untrusted source to pathname generation, which a bad idea anyhow. Most shell scripts do not work on untrusted data. secteam@ says no advisory is necessary. PR: bin/148733 Reported by: Changming Sun snnn119 at gmail com MFC after: 10 days
* sh: Forget about terminated background processes sooner.jilles2010-06-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Unless $! has been referenced for a particular job or $! still contains that job's pid, forget about it after it has terminated. If $! has been referenced, remember the job until the wait builtin has reported its completion (either with the pid as parameter or without parameters). In interactive mode, jobs are forgotten after termination has been reported, which happens before primary prompts and through the jobs builtin. Even then, though, remember a job if $! has been referenced. This is similar to what is suggested by POSIX and should fix most memory leaks (which also tend to cause sh to use more CPU time) with long running scripts that start background jobs. Caveats: * Repeatedly referencing $! without ever doing 'wait', like while :; do foo & echo started foo: $!; sleep 60; done will still use a lot of memory and CPU time in the long run. * The jobs and jobid builtins do not cause a job to be remembered for longer like expanding $! does. PR: bin/55346
* sh: Fix pathname expansion with quoted slashes like *\/.jilles2010-05-111-9/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These are git commits 36f0fa8fcbc8c7b2b194addd29100fb40e73e4e9 and d6d06ff5c2ea0fa44becc5ef4340e5f2f15073e4 in dash. Because this is the first code I'm importing from dash to expand.c, add the Herbert Xu copyright notice which is in dash's expand.c. When pathname expanding *\/, the CTLESC representing the quoted state was erroneously taken as part of the * pathname component. This CTLESC was then seen by the pattern matching code as escaping the '\0' terminating the string. The code is slightly different because dash converts the CTLESC characters to backslashes and removes all the other CTL* characters to allow substituting glob(3). The effect of the bug was also slightly different from dash (where nothing matched at all). Because a CTLESC can escape a '\0' in some way, whether files were included despite the bug depended on memory that should not be read. In particular, on many machines /*\/ expanded to a strict subset of what /*/ expanded to. Example: echo /*"/null" This should print /dev/null, not /*/null. PR: bin/146378 Obtained from: dash
* sh: Use stalloc for arith variable names.jilles2010-04-251-1/+3
| | | | | This is simpler than the custom memory tracker I added earlier, and is also needed by the dash arith code I plan to import.
* sh: Do tilde expansion in substitutions.jilles2010-04-031-3/+3
| | | | | | | | | | | | | This applies to word in ${v-word}, ${v+word}, ${v=word}, ${v?word} (which inherits quoting from the outside) and in ${v%word}, ${v%%word}, ${v#word}, ${v##word} (which does not inherit any quoting). In all cases tilde expansion is only attempted at the start of word, even if word contains spaces. This agrees with POSIX and other shells. This is the last part of the patch tested in the exp-run. Exp-run done by: erwin (with some other sh(1) changes)
* sh: Allow quoting pattern match characters in ${v%pat} and ${v#pat}.jilles2010-04-031-1/+3
| | | | | | | | Note that this depends on r206145 for allowing pattern match characters to have their special meaning inside a double-quoted expansion like "${v%pat}". PR: bin/117748 Exp-run done by: erwin (with some other sh(1) changes)
* sh: Fix some bugs with backquoted builtins:jilles2010-01-011-1/+1
| | | | | | | | | | | | - correctly handle error output in $(builtin 2>&1), clarify out1/out2 vs output/errout in the code - treat all builtins as regular builtins so errors do not abort the shell and variable assignments do not persist - respect the caller's INTOFF Some bugs still exist: - expansion errors may still abort the shell - some side effects of expansions and builtins persist
* sh: Various warning fixes (from WARNS=6 NO_WERROR=1):jilles2009-12-271-2/+1
| | | | | | | - const - initializations to silence -Wuninitialized (it was safe anyway) - remove nested extern declarations - rename "index" locals to "idx"
* sh: Do not consider a tilde-prefix with expansions in it.jilles2009-12-251-2/+7
| | | | | | | | | | That is, do not do tilde expansion if any of the CTL* bytes (\201-\210), not only CTLESC and CTLQUOTEMARK, are encountered. Such an expansion would look up a user name with sh's internal representation. The parser does not currently distinguish between backslashed and unbackslashed \201-\210, so tilde expansion of user names with these bytes in them is not so easy to fix.
* sh: Constify various strings.jilles2009-12-241-6/+6
| | | | | Most of this is adding const keywords, but setvar() in var.c had to be changed somewhat more.
* sh: Exempt $@ and $* from set -ujilles2009-10-241-1/+1
| | | | | | | | This seems more useful and will likely be in the next POSIX standard. Also document more precisely in the man page what set -u does (note that $@, $* and $! are the only special parameters that can ever be unset, all the others are always set, although they may be empty).
* Fix some weirdnesses in the NetBSD IFS code,jilles2009-06-251-2/+3
| | | | | | | | in particular "$@"$ifschar if the final positional parameter is empty. With the NetBSD code, adding the $ifschar removes a parameter. PR: standards/79067 Approved by: ed (mentor) (implicit)
* Improve IFS expansion using code from NetBSD.jilles2009-06-251-61/+80
| | | | | | | | We now pass the ifs.sh testsuite. PR: standards/79067 Approved by: ed (mentor) (implicit) Obtained from: NetBSD
* Expand $LINENO to the current line number. This is required by SUSv3's "Userstefanf2008-05-151-1/+8
| | | | | | | | | Portability Utilities" option. Often configure scripts generated by the autotools test if $LINENO works and refuse to use /bin/sh if not. Package test run by: pav
* Sigh, when reapplying the patch to HEAD, I somehow forgot to commit this file.stefanf2008-04-281-6/+3
| | | | Reported by: Jaakko Heinonen
* Fix expanding of quoted positional parameters in case patterns.stefanf2006-11-071-6/+6
| | | | | | Obtained from: NetBSD (expand.c 1.58 and 1.59) Submitted by: Paul Jarc PR: 56147
* When parsing an invalid parameter expansion (eg. ${} or ${foo@bar}) do notstefanf2006-11-051-0/+5
| | | | | | | | | | | issue a syntax error immediately but save the information that it is erroneous for later when the parameter expansion is actually done. This means eg. "false && ${}" will not generate an error which seems to be required by POSIX. Include the invalid parameter expansion in the error message (sometimes abbreviated with ... because recovering it would require a lot of code). PR: 105078 Submitted by: emaste
* Remove some white space at EOL.schweikh2006-02-041-8/+8
|
* Various small code cleanups resulting from a code reviewingrse2005-09-061-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and linting procedure: 1. Remove useless sub-expression: - if (*start || (!ifsspc && start > string && (nulonly || 1))) { + if (*start || (!ifsspc && start > string)) { The sub-expression "(nulonly || 1)" always evaluates to true and according to CVS logs seems to be just a left-over from some debugging and introduced by accident. Removing the sub-expression doesn't change semantics and a code inspection showed that the variable "nulonly" is also not necessary here in any way (and the expression would require fixing instead of removing). 2. Remove dead code: - if (backslash && c == '\\') { - if (read(STDIN_FILENO, &c, 1) != 1) { - status = 1; - break; - } - STPUTC(c, p); - } else if (ap[1] != NULL && strchr(ifs, c) != NULL) { + if (ap[1] != NULL && strchr(ifs, c) != NULL) { Inspection of the control and data flow showed that variable "backslash" is always false (0) when the "if"-expression is evaluated, hence the whole block is effectively dead code. Additionally, the skipping of characters after a backslash is already performed correctly a few lines above, so this code is also not needed at all. According to the CVS logs and the ASH 0.2 sources, this code existed in this way already since its early days. 3. Cleanup Style: - ! trap[signo][0] == '\0' && + ! (trap[signo][0] == '\0') && The expression wants to ensure the trap is not assigned the empty string. But the "!" operator has higher precedence than "==", so the comparison should be put into parenthesis to form the intended way of expression. Nevertheless the code was effectively not really broken as both particular NUL comparisons are semantically equal, of course. But the parenthesized version is a lot more intuitive. 4. Remove shadowing variable declaration: - char *q; The declaration of symbol "q" hides another identical declaration of "q" in the same context. As the other "q" is already reused multiple times and also can be reused again without negative side-effects, just remove the shadowing declaration. 5. Just small cosmetics: - if (ifsset() != 0) + if (ifsset()) The ifsset() macro is already coded by returning the boolean result of a comparison operator, so no need to compare this boolean result again against a numerical value. This also aligns the macros usage to the remaining existing code. Reviewed by: stefanf@
* Fix a bug when shell expansion is done against dangling symlinks, bydelphij2005-07-071-1/+1
| | | | | | | | | | | | converting the stat() call to a lstat() call, which will cover the situation. One can exercise this bug by referring a dangling link with something like */the-link. Approved by: re (scottl) Submitted by: Simon 'corecode' Schubert [corecode fs ei tum de] Obtained from: NetBSD via DragonFlyBSD (NetBSD rev. 1.51 and DragonFly rev. 1.6) MFC After: 3 days
* Remove clause 3 from the UCB licenses.markm2004-04-061-4/+0
| | | | OK'ed by: imp, core
OpenPOWER on IntegriCloud