coherence.upnp.core.utils (module)

Utils

Set of utilities to help process the data and the assets of the Cohen3 project. It includes several methods which covers different fields, here are grouped by his functionality:

• encode/decode strings:

  • to_string()

  • to_bytes()

• parse xml/html data:

  • parse_xml()

  • parse_http()

  • de_chunk_payload()

• get ip/host:

  • get_ip_address()

  • get_host_address()

• get/download page related:

  • getPage()

  • downloadPage()

  • myHTTPPageGetter

  • HeaderAwareHTTPClientFactory

• proxy clients and resources:

  • ProxyClient

  • ProxyClientFactory

  • ReverseProxyResource

  • ReverseProxyUriResource

• file assets:

  • StaticFile

  • BufferFile

  • BufferFileTransfer

• date/time operations:

  • _tz

  • _CET

  • _CEST

  • datefaker()

  • _bdates

• python 2to3 compatibility methods:

  • cmp()

to_string(x)

This method is a helper function that takes care of converting into a string any string or bytes string or integer. This is useful for decoding twisted responses into the default python 3 string encoding or to get a string representation of an object.

New in version 0.8.2.

Note

If the argument passed is not of type str, bytes or int, it will try to get the string representation of the object.

Warning

This is similar to to_bytes() but with the difference that the returned result it will be always a string.

to_bytes(x)

This method is a helper function that takes care of converting a string or string of bytes into bytes, needed for most of the write operations for twisted responses. It is useful when we don’t know the type of the processed string.

New in version 0.8.2.

Changed in version 0.8.3: Errors will be bypassed with a warning

Note

If the argument passed is not of type str or bytes, it will be converted to his string representation and then it will be converted into bytes.

Warning

If, while encoding, some error is encountered, it will be bypassed and user will be notified with a log warning. The conflicting character will be replaced for the symbol “?” (U+FFFD)

means_true(value)

Transform a value representing a boolean into a boolean.

The valid expressions are:

• True or ‘True’

• 1 or ‘1’

• ‘yes’ or ‘ok’

Note

the string expressions are not case sensitive

generalise_boolean(value)

standardize the different boolean incarnations

transform anything that looks like a ‘True’ into a ‘1’, and everything else into a ‘0’

generalize_boolean(value)

standardize the different boolean incarnations

transform anything that looks like a ‘True’ into a ‘1’, and everything else into a ‘0’

parse_xml(data, encoding='utf-8', dump_invalid_data=False)

Takes an xml string and returns an XML element hierarchy

parse_with_lxml(data, encoding='utf-8')

Takes an xml string or a response as argument and returns a root tree. This method is similar to parse_xml() but here we use the lxml module and a custom parser method to return an lxml’s ElementTree object.

New in version 0.8.3.

Note

This parser allow us to parse successfully some responses which contain encoding defined (ex: soap messages) and also has the ability to parse a broken xml. This method could be useful to parse some small pieces of html code into an xml tree in order to extract some info.

parse_http_response(data)

Takes a response as argument and returns a tuple: cmd, headers

The first value of the tuple (cmd) will contain the server response and the second one the headers.

Note

don’t try to get the body, there are responses without

get_ip_address(ifname)

Determine the IP address by interface name

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/439094 (c) Paul Cannon Uses the Linux SIOCGIFADDR ioctl to find the IP address associated with a network interface, given the name of that interface, e.g. “eth0”. The address is returned as a string containing a dotted quad.

Updated to work on BSD. OpenBSD and OSX share the same value for SIOCGIFADDR, and its likely that other BSDs do too.

Updated to work on Windows, using the optional Python module netifaces http://alastairs-place.net/netifaces/

Thx Lawrence for that patch!

get_host_address()

try to get determine the interface used for the default route, as this is most likely the interface we should bind to (on a single homed host!)

de_chunk_payload(response)

class Request(*args, **kw)

Bases: twisted.web.server.Request

Custom implementation of twisted.web.server.Request which takes care of process data for our needs.

process()

Process a request.

class Site(resource, requestFactory=None, *args, **kwargs)

Bases: twisted.web.server.Site

Custom implementation of twisted.web.server.Site

@param resource: The root of the resource hierarchy. All request

traversal for requests received by this factory will begin at this resource.

@type resource: L{IResource} provider @param requestFactory: Overwrite for default requestFactory. @type requestFactory: C{callable} or C{class}.

@see: L{twisted.web.http.HTTPFactory.__init__}

noisy = False

requestFactory

alias of Request

startFactory()

Set up request logging if necessary.

class ProxyClient(command, rest, version, headers, data, father)

Bases: twisted.web.proxy.ProxyClient, coherence.log.LogAble

handleStatus(version, code, message)

Called when the status-line is received.

@param version: e.g. ‘HTTP/1.0’ @param status: e.g. ‘200’ @type status: C{bytes} @param message: e.g. ‘OK’

handleHeader(key, value)

Called every time a header is received.

handleResponsePart(buffer)

class ProxyClientFactory(command, rest, version, headers, data, father)

Bases: twisted.web.proxy.ProxyClientFactory

protocol

alias of twisted.web.proxy.ProxyClient

class ReverseProxyResource(host, port, path, reactor=<twisted.internet.epollreactor.EPollReactor object>)

Bases: twisted.web.proxy.ReverseProxyResource

Resource that renders the results gotten from another server.

Put this resource in the tree to cause everything below it to be relayed to a different server.

Parameters

• host (str) – the host of the web server to proxy.

• port (int) – the port of the web server to proxy.

• path (str) – the base path to fetch data from. Note that you shouldn’t put any trailing slashes in it, it will be added automatically in request. For example, if you put B{/foo}, a request on B{/bar} will be proxied to B{/foo/bar}.

• reactor (twisted.internet.interfaces.IReactorTCP) – the reactor used to create connections.

proxyClientFactoryClass

proxyClientFactoryClass (twisted.web.proxy.ProxyClientFactory): a proxy client factory class, used to create new connections.

alias of ProxyClientFactory

__init__(host, port, path, reactor=<twisted.internet.epollreactor.EPollReactor object>)

Parameters

• host (str) – the host of the web server to proxy.

• port (int) – the port of the web server to proxy.

• path (str) – the base path to fetch data from. Note that you shouldn’t put any trailing slashes in it, it will be added automatically in request. For example, if you put B{/foo}, a request on B{/bar} will be proxied to B{/foo/bar}.

• reactor (twisted.internet.interfaces.IReactorTCP) – the reactor used to create connections.

getChild(path, request)

Create and return a proxy resource with the same proxy configuration as this one, except that its path also contains the segment given by C{path} at the end.

render(request)

Render a request by forwarding it to the proxied server.

resetTarget(host, port, path, qs='')

class ReverseProxyUriResource(uri, reactor=<twisted.internet.epollreactor.EPollReactor object>)

Bases: coherence.upnp.core.utils.ReverseProxyResource

uri = None

resetUri(uri)

class myHTTPPageGetter

Bases: twisted.web.client.HTTPPageGetter

followRedirect = True

class HeaderAwareHTTPClientFactory(url, method=b'GET', postdata=None, headers=None, agent=b'Twisted PageGetter', timeout=0, cookies=None, followRedirect=True, redirectLimit=20, afterFoundGet=False)

Bases: twisted.web.client.HTTPClientFactory

protocol

alias of myHTTPPageGetter

noisy = False

buildProtocol(addr)

Create an instance of a subclass of Protocol.

The returned instance will handle input on an incoming server connection, and an attribute “factory” pointing to the creating factory.

Alternatively, L{None} may be returned to immediately close the new connection.

Override this method to alter how Protocol instances get created.

@param addr: an object implementing L{twisted.internet.interfaces.IAddress}

page(page)

getPage(url, contextFactory=None, *args, **kwargs)

Download a web page as a string.

Download a page. Return a deferred, which will callback with a page (as a string) or errback with a description of the error.

See twisted.web.client.HTTPClientFactory to see what extra args can be passed.

Note

This function is like twisted.web.client.getPage, except it uses our HeaderAwareHTTPClientFactory instead of HTTPClientFactory and sets the user agent.

downloadPage(url, file, contextFactory=None, *args, **kwargs)

Download a web page to a file.

Parameters

• url (str or bytes) – target url to download.

• file (str or file-like object) – path to file on filesystem, or a file-like object.

Note

See twisted.web.client.HTTPDownloader to see what extra args can be passed.

class BufferFile(path, target_size=0, *args)

Bases: twisted.web.static.File

Custom implementation of twisted.web.static.File and modified accordingly to the patch by John-Mark Gurney ( http://resnet.uoregon.edu/~gurney_j/jmpc/dist/twisted.web.static.patch)

Note

See twisted.web.static.File to see what extra args can be passed.

render(request)

Render a given resource. See L{IResource}’s render method.

I delegate to methods of self with the form ‘render_METHOD’ where METHOD is the HTTP that was used to make the request. Examples: render_GET, render_HEAD, render_POST, and so on. Generally you should implement those methods instead of overriding this one.

render_METHOD methods are expected to return a byte string which will be the rendered page, unless the return value is C{server.NOT_DONE_YET}, in which case it is this class’s responsibility to write the results using C{request.write(data)} and then call C{request.finish()}.

Old code that overrides render() directly is likewise expected to return a byte string or NOT_DONE_YET.

@see: L{IResource.render}

class BufferFileTransfer(file, size, request)

Bases: object

A class to represent the transfer of a file over the network.

request = None

resumeProducing()

pauseProducing()

stopProducing()

class _tz

Bases: datetime.tzinfo

Custom base class for time zone info classes.

utcoffset(dt)

datetime -> timedelta showing offset from UTC, negative values indicating West of UTC

tzname(dt)

datetime -> string name of time zone.

dst(dt)

datetime -> DST offset in minutes east of UTC.

class _CET

Bases: coherence.upnp.core.utils._tz

Custom class for time zone representing Central European Time.

_offset = datetime.timedelta(0, 3600)

_name = 'CET'

class _CEST

Bases: coherence.upnp.core.utils._tz

Custom class for time zone representing Central European Summer Time.

_offset = datetime.timedelta(0, 7200)

_name = 'CEST'

datefaker()

Choose a random datetime from _bdates

Note

Used inside class Object, method toElement()

cmp(x, y)

Replacement for built-in function cmp that was removed in Python 3

Compare the two objects x and y and return an integer according to the outcome. The return value is negative if x < y, zero if x == y and strictly positive if x > y.

Parameters

• x (object) – An object

• y (object) – Another object to compare with x