furl

## furl is a small Python library that makes parsing and
modifying URLs easy. Python's standard [urllib](https://docs.python.org/3/library/urllib.html) and [urlparse](https://docs.python.org/3/library/urllib.parse.html) modules provide a number of URL related functions, but using these functions to perform common URL operations proves tedious. Furl makes parsing and modifying URLs easy. Furl is well tested, [Unlicensed](http://unlicense.org/) in the public domain, and supports Python 3 and PyPy3. Furl is maintained by [Alex Cochran](https://github.com/alexcochran), with support from the confidential computing folks at [ Lunal](https://lunal.dev/). ## Usage Code time: Paths and query arguments are easy. Really easy. ```python >>> from furl import furl >>> f = furl('http://www.google.com/?one=1&two=2') >>> f /= 'path' >>> del f.args['one'] >>> f.args['three'] = '3' >>> f.url 'http://www.google.com/path?two=2&three=3' ``` Or use furl's inline modification methods. ```python >>> furl('http://www.google.com/?one=1').add({'two':'2'}).url 'http://www.google.com/?one=1&two=2' >>> furl('http://www.google.com/?one=1&two=2').set({'three':'3'}).url 'http://www.google.com/?three=3' >>> furl('http://www.google.com/?one=1&two=2').remove(['one']).url 'http://www.google.com/?two=2' ``` Encoding is handled for you. Unicode, too. ```python >>> f = furl('http://www.google.com/') >>> f.path = 'some encoding here' >>> f.args['and some encoding'] = 'here, too' >>> f.url 'http://www.google.com/some%20encoding%20here?and+some+encoding=here,+too' >>> f.set(host=u'ドメイン.テスト', path=u'джк', query=u'☃=☺') >>> f.url 'http://xn--eckwd4c7c.xn--zckzah/%D0%B4%D0%B6%D0%BA?%E2%98%83=%E2%98%BA' ``` Fragments also have a path and a query. ```python >>> f = furl('http://www.google.com/') >>> f.fragment.path.segments = ['two', 'directories'] >>> f.fragment.args = {'one': 'argument'} >>> f.url 'http://www.google.com/#two/directories?one=argument' ``` ## Installation Installing furl with pip is easy. ``` $ pip install furl ``` ## API * [Basics](#basics) * [Scheme, Username, Password, Host, Port, Network Location, and Origin](#scheme-username-password-host-port-network-location-and-origin) * [Path](#path) * [Modification](#modification) * [Query](#query) * [Modification](#modification-1) * [Parameters](#parameters) * [Fragment](#fragment) * [Encoding](#encoding) * [Inline modification](#inline-modification) * [Miscellaneous](#miscellaneous) ### Basics furl objects let you access and modify the various components of a URL. ``` scheme://username:password@host:port/path?query#fragment ``` * __scheme__ is the scheme string (all lowercase) or None. None means no scheme. An empty string means a protocol relative URL, like `//www.google.com`. * __username__ is the username string for authentication. * __password__ is the password string for authentication with __username__. * __host__ is the domain name, IPv4, or IPv6 address as a string. Domain names are all lowercase. * __port__ is an integer or None. A value of None means no port specified and the default port for the given __scheme__ should be inferred, if possible (e.g. port 80 for the scheme `http`). * __path__ is a Path object comprised of path segments. * __query__ is a Query object comprised of key:value query arguments. * __fragment__ is a Fragment object comprised of a Path object and Query object separated by an optional `?` separator. ### Scheme, Username, Password, Host, Port, Network Location, and Origin __scheme__, __username__, __password__, and __host__ are strings or None. __port__ is an integer or None. ```python >>> f = furl('http://user:[email protected]:99/') >>> f.scheme, f.username, f.password, f.host, f.port ('http', 'user', 'pass', 'www.google.com', 99) ``` furl infers the default port for common schemes. ```python >>> f = furl('https://secure.google.com/') >>> f.port 443 >>> f = furl('unknown://www.google.com/') >>> print(f.port) None ``` __netloc__ is the string combination of __username__, __password__, __host__, and __port__, not including __port__ if it's None or the default port for the provided __scheme__. ```python >>> furl('http://www.google.com/').netloc 'www.google.com' >>> furl('http://www.google.com:99/').netloc 'www.google.com:99' >>> furl('http://user:[email protected]:99/').netloc 'user:[email protected]:99' ``` __origin__ is the string combination of __scheme__, __host__, and __port__, not including __port__ if it's None or the default port for the provided __scheme__. ```python >>> furl('http://www.google.com/').origin 'http://www.google.com' >>> furl('http://www.google.com:99/').origin 'http://www.google.com:99' ``` ### Path URL paths in furl are Path objects that have __segments__, a list of zero or more path segments that can be modified directly. Path segments in __segments__ are percent-decoded and all interaction with __segments__ should take place with percent-decoded strings. ```python >>> f = furl('http://www.google.com/a/large%20ish/path') >>> f.path Path('/a/large ish/path') >>> f.path.segments ['a', 'large ish', 'path'] >>> str(f.path) '/a/large%20ish/path' ``` #### Modification ```python >>> f.path.segments = ['a', 'new', 'path', ''] >>> str(f.path) '/a/new/path/' >>> f.path = 'o/hi/there/with%20some%20encoding/' >>> f.path.segments ['o', 'hi', 'there', 'with some encoding', ''] >>> str(f.path) '/o/hi/there/with%20some%20encoding/' >>> f.url 'http://www.google.com/o/hi/there/with%20some%20encoding/' >>> f.path.segments = ['segments', 'are', 'maintained', 'decoded', '^`<>[]"#/?'] >>> str(f.path) '/segments/are/maintained/decoded/%5E%60%3C%3E%5B%5D%22%23%2F%3F' ``` A path that starts with `/` is considered absolute, and a Path can be absolute or not as specified (or set) by the boolean attribute __isabsolute__. URL Paths have a special restriction: they must be absolute if a __netloc__ (username, password, host, and/or port) is present. This restriction exists because a URL path must start with `/` to separate itself from the __netloc__, if present. Fragment Paths have no such limitation and __isabsolute__ and can be True or False without restriction. Here's a URL Path example that illustrates how __isabsolute__ becomes True and read-only in the presence of a __netloc__. ```python >>> f = furl('/url/path') >>> f.path.isabsolute True >>> f.path.isabsolute = False >>> f.url 'url/path' >>> f.host = 'blaps.ru' >>> f.url 'blaps.ru/url/path' >>> f.path.isabsolute True >>> f.path.isabsolute = False Traceback (most recent call last): ... AttributeError: Path.isabsolute is True and read-only for URLs with a netloc (a username, password, host, and/or port). URL paths must be absolute if a netloc exists. >>> f.url 'blaps.ru/url/path' ``` Conversely, the __isabsolute__ attribute of Fragment Paths isn't bound by the same read-only restriction. URL fragments are always prefixed by a `#` character and don't need to be separated from the __netloc__. ```python >>> f = furl('http://www.google.com/#/absolute/fragment/path/') >>> f.fragment.path.isabsolute True >>> f.fragment.path.isabsolute = False >>> f.url 'http://www.google.com/#absolute/fragment/path/' >>> f.fragment.path.isabsolute = True >>> f.url 'http://www.google.com/#/absolute/fragment/path/' ``` A path that ends with `/` is considered a directory, and otherwise considered a file. The Path attribute __isdir__ returns True if the path is a directory, False otherwise. Conversely, the attribute __isfile__ returns True if the path is a file, False otherwise. ```python >>> f = furl('http://www.google.com/a/directory/') >>> f.path.isdir True >>> f.path.isfile False >>> f = furl('http://www.google.com/a/file') >>> f.path.isdir False >>> f.path.isfile True ``` A path can be normalized with __normalize()__, and __normalize()__ returns the Path object for method chaining. ```python >>> f = furl('http://www.google.com////a/./b/lolsup/../c/') >>> f.path.normalize() >>> f.url 'http://www.google.com/a/b/c/' ``` Path segments can also be appended with the slash operator, like with [pathlib.Path](https://docs.python.org/3/library/pathlib.html#operators). ```python >>> from __future__ import division # For Python 2.x. >>> >>> f = furl('path') >>> f.path /= 'with' >>> f.path = f.path / 'more' / 'path segments/' >>> f.url '/path/with/more/path%20segments/' ``` For a dictionary representation of a path, use __asdict()__. ```python >>> f = furl('http://www.google.com/some/enc%20oding') >>> f.path.asdict() { 'encoded': '/some/enc%20oding', 'isabsolute': True, 'isdir': False, 'isfile': True, 'segments': ['some', 'enc oding'] } ``` ### Query URL queries in furl are Query objects that have __params__, a one dimensional [ordered multivalue dictionary](https://github.com/gruns/orderedmultidict) of query keys and values. Query keys and values in __params__ are percent-decoded and all interaction with __params__ should take place with percent-decoded strings. ```python >>> f = furl('http://www.google.com/?one=1&two=2') >>> f.query Query('one=1&two=2') >>> f.query.params omdict1D([('one', '1'), ('two', '2')]) >>> str(f.query) 'one=1&two=2' ``` furl objects and Fragment objects (covered below) contain a Query object, and __args__ is provided as a shortcut on these objects to access __query.params__. ```python >>> f = furl('http://www.google.com/?one=1&two=2') >>> f.query.params omdict1D([('one', '1'), ('two', '2')]) >>> f.args omdict1D([('one', '1'), ('two', '2')]) >>> f.args is f.query.params True ``` #### Modification __params__ is a one dimensional [ordered multivalue dictionary](https://github.com/gruns/orderedmultidict) that maintains method parity with Python's standard dictionary. ```python >>> f.query = 'silicon=14&iron=26&inexorable%20progress=vae%20victus' >>> f.query.params omdict1D([('silicon', '14'), ('iron', '26'), ('inexorable progress', 'vae victus')]) >>> del f.args['inexorable progress'] >>> f.args['magnesium'] = '12' >>> f.args omdict1D([('silicon', '14'), ('iron', '26'), ('magnesium', '12')]) ``` __params__ can also store multiple values for the same key because it's a multivalue dictionary. ```python >>> f = furl('http://www.google.com/?space=jams&space=slams') >>> f.args['space'] 'jams' >>> f.args.getlist('space') ['jams', 'slams'] >>> f.args.addlist('repeated', ['1', '2', '3']) >>> str(f.query) 'space=jams&space=slams&repeated=1&repeated=2&repeated=3' >>> f.args.popvalue('space') 'slams' >>> f.args.popvalue('repeated', '2') '2' >>> str(f.query) 'space=jams&repeated=1&repeated=3' ``` __params__ is one dimensional. If a list of values is provided as a query value, that list is interpreted as multiple values. ```python >>> f = furl() >>> f.args['repeated'] = ['1', '2', '3'] >>> f.add(args={'space':['jams', 'slams']}) >>> str(f.query) 'repeated=1&repeated=2&repeated=3&space=jams&space=slams' ``` This makes sense: URL queries are inherently one dimensional -- query values can't have native subvalues. See the [orderedmultimdict](https://github.com/gruns/orderedmultidict) documentation for more information on interacting with the ordered multivalue dictionary __params__. #### Parameters To produce an empty query argument, like `http://sprop.su/?param=`, set the argument's value to the empty string. ```python >>> f = furl('http://sprop.su') >>> f.args['param'] = '' >>> f.url 'http://sprop.su/?param=' ``` To produce an empty query argument without a trailing `=`, use `None` as the parameter value. ```python >>> f = furl('http://sprop.su') >>> f.args['param'] = None >>> f.url 'http://sprop.su/?param' ``` __encode(delimiter='&', quote_plus=True, dont_quote='')__ can be used to encod