doublestar

doublestar

Path pattern matching and globbing supporting doublestar (**) patterns.

About

Upgrading?

doublestar is a [golang] implementation of path pattern matching and globbing with support for "doublestar" (aka globstar: **) patterns.

doublestar patterns match files and directories recursively. For example, if you had the following directory structure:

grandparent
`-- parent
    |-- child1
    `-- child2

You could find the children with patterns such as: **/child*, grandparent/**/child?, **/parent/*, or even just ** by itself (which will return all files and directories recursively).

Bash's globstar is doublestar's inspiration and, as such, works similarly. Note that the doublestar must appear as a path component by itself. A pattern such as /path** is invalid and will be treated the same as /path*, but /path*/** should achieve the desired result. Additionally, /path/** will match all directories and files under the path directory, but /path/**/ will only match directories.

v4 is a complete rewrite with a focus on performance. Additionally, [doublestar] has been updated to use the new [io/fs] package for filesystem access. As a result, it is only supported by [golang] v1.16+.

Installation

doublestar can be installed via go get:

go get github.com/bmatcuk/doublestar/v4

To use it in your code, you must import it:

import "github.com/bmatcuk/doublestar/v4"

Usage

ErrBadPattern

doublestar.ErrBadPattern

Returned by various functions to report that the pattern is malformed. At the moment, this value is equal to path.ErrBadPattern, but, for portability, this equivalence should probably not be relied upon.

Match

func Match(pattern, name string) (bool, error)

Match returns true if name matches the file name pattern ([see "patterns"]). name and pattern are split on forward slash (/) characters and may be relative or absolute.

Match requires pattern to match all of name, not just a substring. The only possible returned error is ErrBadPattern, when pattern is malformed.

Note: this is meant as a drop-in replacement for path.Match() which always uses '/' as the path separator. If you want to support systems which use a different path separator (such as Windows), what you want is PathMatch(). Alternatively, you can run filepath.ToSlash() on both pattern and name and then use this function.

Note: users should not count on the returned error, doublestar.ErrBadPattern, being equal to path.ErrBadPattern.

MatchUnvalidated

func MatchUnvalidated(pattern, name string) bool

MatchUnvalidated can provide a small performance improvement if you don't care about whether or not the pattern is valid (perhaps because you already ran ValidatePattern). Note that there's really only one case where this performance improvement is realized: when pattern matching reaches the end of name before reaching the end of pattern, such as Match("a/b/c", "a").

PathMatch

func PathMatch(pattern, name string) (bool, error)

PathMatch returns true if name matches the file name pattern ([see "patterns"]). The difference between Match and PathMatch is that PathMatch will automatically use your system's path separator to split name and pattern. On systems where the path separator is '\', escaping will be disabled.

Note: this is meant as a drop-in replacement for filepath.Match(). It assumes that both pattern and name are using the system's path separator. If you can't be sure of that, use filepath.ToSlash() on both pattern and name, and then use the Match() function instead.

PathMatchUnvalidated

func PathMatchUnvalidated(pattern, name string) bool

PathMatchUnvalidated can provide a small performance improvement if you don't care about whether or not the pattern is valid (perhaps because you already ran ValidatePattern). Note that there's really only one case where this performance improvement is realized: when pattern matching reaches the end of name before reaching the end of pattern, such as Match("a/b/c", "a").

GlobOption

Options that may be passed to Glob, GlobWalk, or FilepathGlob. Any number of options may be passed to these functions, and in any order, as the last argument(s).

WithCaseInsensitive()

WithCaseInsensitive is an option that can be passed to Glob, GlobWalk, or FilepathGlob. If passed, doublestar will treat all alphabetic characters as case insensitive (i.e. "a" in the pattern would match "a" or "A"). This is useful for platforms like Windows where paths are case insensitive by default.

WithFailOnIOErrors()

If passed, doublestar will abort and return IO errors when encountered. Note that if the glob pattern references a path that does not exist (such as nonexistent/path/*), this is not considered an IO error: it is considered a pattern with no matches.

WithFailOnPatternNotExist()

If passed, doublestar will abort and return doublestar.ErrPatternNotExist if the pattern references a path that does not exist before any meta characters such as nonexistent/path/*. Note that alts (ie, {...}) are expanded before this check. In other words, a pattern such as {a,b}/* may fail if either a or b do not exist but */{a,b} will never fail because the star may match nothing.

WithFilesOnly()

If passed, doublestar will only return "files" from Glob, GlobWalk, or FilepathGlob. In this context, "files" are anything that is not a directory or a symlink to a directory.

Note: if combined with the WithNoFollow option, symlinks to directories will be included in the result since no attempt is made to follow the symlink.

WithNoFollow()

If passed, doublestar will not follow symlinks while traversing the filesystem. However, due to io/fs's very poor support for querying the filesystem about symlinks, there's a caveat here: if part of the pattern before any meta characters contains a reference to a symlink, it will be followed. For example, a pattern such as path/to/symlink/* will be followed assuming it is a valid symlink to a directory. However, from this same example, a pattern such as path/to/** will not traverse the symlink, nor would path/*/symlink/*

Note: if combined with the WithFilesOnly option, symlinks to directories will be included in the result since no attempt is made to follow the symlink.

WithNoHidden()

If passed, doublestar will not match hidden files and directories (those starting with a dot) when using wildcards. This follows traditional shell glob behavior where * or a ? at the start will not match dotfiles by default.

Hidden files can still be matched by explicitly including them in the pattern. For example, .* will match hidden files, and .config/** will match files inside the .config directory.

The rule is:

• For **: do not descend into hidden directories
• For * or a pattern starting with ?: do not match dotfiles or directories

On Windows, doublestar will check the file attributes and avoid hidden files and directories this way, instead of matching the filename. Therefore, any pattern with a * or ? could potentially match a hidden file/directory.

Glob

func Glob(fsys fs.FS, pattern string, opts ...GlobOption) ([]string, error)

Glob returns the names of all files matching pattern or nil if there is no matching file. The syntax of patterns is the same as in Match(). The pattern may describe hierarchical names such as usr/*/bin/ed.

Glob ignores file system errors such as I/O errors reading directories by default. The only possible returned error is ErrBadPattern, reporting that the pattern is malformed.

To enable aborting on I/O errors, the WithFailOnIOErrors option can be passed.

Note: this is meant as a drop-in replacement for io/fs.Glob(). Like io/fs.Glob(), this function assumes that your pattern uses / as the path separator even if that's not correct for your OS (like Windows). If you aren't sure if that's the case, you can use filepath.ToSlash() on your pattern before calling Glob().

Like io/fs.Glob(), patterns containing /./, /../, or starting with / will return no results and no errors. This seems to be a conscious decision, even if counter-intuitive. You can use [SplitPattern] to divide a pattern into a base path (to initialize an FS object) and pattern.

Note: users should not count on the returned error, doublestar.ErrBadPattern, being equal to path.ErrBadPattern.

GlobWalk

type GlobWalkFunc func(path string, d fs.DirEntry) error

func GlobWalk(fsys fs.FS, pattern string, fn GlobWalkFunc, opts ...GlobOption) error

GlobWalk calls the callback function fn for every file matching pattern. The syntax of pattern is the same as in Match() and the behavior is the same as Glob(), with regard to limitations (such as patterns containing /./, /../, or starting with /). The pattern may describe hierarchical names such as usr/*/bin/ed.

GlobWalk may have a small performance benefit over Glob if you do not need a slice of matches because it can avoid allocating memory for the matches. Additionally, GlobWalk gives you access to the fs.DirEntry objects for each match, and lets you quit early by returning a non-nil error from your callback function. Like io/fs.WalkDir, if your callback returns SkipDir, GlobWalk will skip the current directory. This means that if the current path is a directory, GlobWalk will not recurse into it. If the current path is not a directory, the rest of the parent directory will be skipped.

GlobWalk ignores file system errors such as I/O errors reading directories by default. GlobWalk may return ErrBadPattern, reporting that the pattern is malformed.

To enable aborting on I/O errors, the WithFailOnIOErrors option can be passed.

Additionally, if the callback function fn returns an error, GlobWalk will exit immediately and return that error.

Like Glob(), this function assumes that your pattern uses / as the path separator even if that's not correct for your OS (like Windows). If you aren't sure if that's the case, you can use filepath.ToSlash() on your pattern before calling GlobWalk().

Note: users should not count on the returned error, doublestar.ErrBadPattern, being equal to path.ErrBadPattern.

FilepathGlob

func FilepathGlob(pattern string, opts ...GlobOption) (matches []string, err error)

FilepathGlob returns the names of all files matching pattern or nil if there is no matching file. The syntax of pattern is the same as in Match(). The pattern may describe hierarchical names such as usr/*/bin/ed.

FilepathGlob ignores file system errors such as I/O errors reading directories by default. The only possible returned error is ErrBadPattern, reporting that the pattern is malformed.

To enable aborting on I/O errors, the WithFailOnIOErrors option can be passed.

Note: FilepathGlob is a convenience function that is meant as a drop-in replacement for path/filepath.Glob() for users who don't need the complication of io/fs. Basically, it:

• Runs filepath.Clean() and ToSlash() on the pattern
• Runs SplitPattern() to get a base path and a pattern to Glob
• Creates an FS object from the base path and Glob()s on the pattern
• Joins the base path with all of the matches from Glob()

Returned paths will use the system's path separator, just like filepath.Glob().

Note: the returned error doublestar.ErrBadPattern is not equal to filepath.ErrBadPattern.

SplitPattern

func SplitPattern(p string) (base, pattern string)

SplitPattern is a utility function. Given a pattern, SplitPattern will return two strings: the first string is everything up to the last slash (/) that