diff --git a/debian/changelog b/debian/changelog index e50f806..2061359 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,220 +1,226 @@ +msrprelay (1.4.0) unstable; urgency=medium + + * Convert to Python3 + + -- Adrian Georgescu Wed, 10 Mar 2021 00:32:32 +0100 + msrprelay (1.3.1) unstable; urgency=medium * Fixed installing signal handlers -- Adrian Georgescu Wed, 15 Apr 2020 15:53:55 +0200 msrprelay (1.3.0) unstable; urgency=medium * Fixed some PEP-8 compliance issues * Initialize the log level in the main script * Updated to the logging API in python-application 2.8.0 * Use the argparse module for parsing command line arguments * Added command line option for memory debugging * Added command line option to enable verbose logging * Adapted to process API changes in python-application 2.8.0 * Added command line option to run as a systemd service * Added code to wait for the network to be available when starting * Display the default configuration directory in the help output * Refactored setup.py * Install the sample configuration file from setup.py * Updated copyright years * Added debian/source/format to MANIFEST.in * Added boring file * Removed duplicate changelog * Removed no longer needed entry in MANIFEST.in * Made MANIFEST.in more explicit and avoid unnecessary prune commands * Updated boring file to ignore generated TLS certificates * Explicitly use python2 in shebang lines * Split the debian dependencies one per line for readability * Removed no longer needed debian pycompat and pyversions files * Removed no longer needed debian version dependency restrictions * Removed no longer needed debian dirs file * Increased debian compatibility level to 11 * Updated debian python build system * Fixed permissions for debian installed configuration file * Silenced lintian warning about missing manual page * Switched from the init script to a systemd service file * Updated debian uploaders * Removed no longer needed debian dependency on lsb-base * Removed unnecessary package name prefixes from some debian files * Install TLS files in /usr/share/msrprelay/tls for the debian package * Install README and setup instructions in debian package docs directory * Create an empty /etc/msrprelay/tls/ directory in the debian package * Removed commented out variable in debian rules * Increased debian standards version to 4.5.0 * Updated copyright years -- Dan Pascu Fri, 14 Feb 2020 13:57:52 +0200 msrprelay (1.2.2) unstable; urgency=medium * Support systemd by redirecting init.d script calls to it if present * Increased debian compatibility level to 9 * Updated debian standards version * Updated debian maintainer -- Dan Pascu Fri, 20 Jan 2017 09:47:44 +0200 msrprelay (1.2.1) unstable; urgency=medium * Adapt to changes in python-gnutls 3.0 * Don't use mutable types as parameters -- Saul Ibarra Thu, 10 Mar 2016 12:05:02 +0100 msrprelay (1.2.0) unstable; urgency=medium * Removed runtime dependency check -- Saul Ibarra Wed, 06 Jan 2016 15:38:45 +0100 msrprelay (1.1.0) unstable; urgency=medium * Fix forwarding non-SEND requests and replies * Make the code more RFC compliant * Code cleanup * Set allow_other_methods to True by default * Drop Python < 2.7 support * Bumped Debian Standards-Version -- Saul Ibarra Wed, 29 Apr 2015 10:59:58 +0200 msrprelay (1.0.8) unstable; urgency=low * Added SIPThor integration * Allow NICKNAME as valid MSRP method -- Saul Ibarra Fri, 25 Jan 2013 15:57:53 +0100 msrprelay (1.0.7) unstable; urgency=low * Fixed matching of DNS names that contain wildcards in X509 certificates * Documented usage of multiple relays per domain * Added credits about NLnet foundation sponsorship -- Adrian Georgescu Tue, 22 Nov 2011 20:24:37 +0100 msrprelay (1.0.6) unstable; urgency=low * Adapted to latest changes in python-application * Reworked Debian packaging -- Saul Ibarra Fri, 10 Jun 2011 09:44:56 +0200 msrprelay (1.0.5) unstable; urgency=low * Use hexdigest.md5 istead of md5 -- Adrian Georgescu Thu, 28 Oct 2010 20:26:45 +0200 msrprelay (1.0.4) unstable; urgency=low * Fixed typo in startup script * Bumped Standards-Version to 3.9.1 * Set default credentials for OpenSIPS database in sample config file * Fixed dependency on $remote_fs in init script -- Adrian Georgescu Wed, 01 Sep 2010 10:52:40 +0200 msrprelay (1.0.3) unstable; urgency=low * Modified code to use the latest facilities from python-application 1.1.5 * Updated minimum version dependency on python-application * Updated list of pruned directories from the source distribution * Restart server after upgrade * Updated readme and install documents -- Adrian Georgescu Tue, 04 Aug 2009 10:20:39 +0200 msrprelay (1.0.2) unstable; urgency=low * Adapted code to work with the latest python-application changes: - use ConfigSetting descriptors instead of _datatypes - replaced startSyslog with start_syslog - use system_config_directory instead of _system_config_directory * Added runtime dependency checks using application.dependency * Small fix in forwarding logic * Improved error messages when TLS certificates are missing or faulty * Added log_level option to configuration file * Fixed debian building warnings * Update sample config file * Moved tasks to wiki tickets * Fixed parameter quoting in WWW-Authenticate and Authentication-Info header * Added relay session_id check for incoming messages * Use os.urandom instead of reading directly from /dev/urandom * Allow refreshing AUTH from client, nothing is actually refreshed * Fixed the first characted of generated transaction-id to be alphanumeric * Correct callback sequence for authentication Deferred * Updated debian dependencies * Updated dependencies in INSTALL * Removed unused imports * Minimized and updated build and runtime dependencies * Removed obsolete python version specifications in debian control file * Added missing files to source distribution * Fixed lintian warning about missing misc:Depends -- Adrian Georgescu Mon, 03 Aug 2009 21:15:03 +0200 msrprelay (1.0.1) unstable; urgency=low * Remove the session from unbound_session when we start trying to make an outbound connection * Added instructions for how to install or build the Debian package -- Adrian Georgescu Thu, 27 Nov 2008 10:44:27 +0100 msrprelay (1.0.0) unstable; urgency=low * Removed per-domain configuration and certificates in favour of detecting the SIP domain from the To-Path. This assumes SRV records are used to look up the MSRP relay. This also elimites the need for using the TLS server name extension. * Removed any references to CAs and CRLs. * Simplified certificate generation. * Cleaned up test script directory. * Added own runtime directory in /var/run to store runtime files. * Added commandline option to specify the name of the config file to read. * Fixed the "memory" backend to support domain names. * Added username@domain and total session bytecount to logging output. * Several miscellaneous fixes based on field experience. -- Ruud Klaver Mon, 08 Sep 2008 19:14:09 +0200 msrprelay (0.9.0) unstable; urgency=low * Initial release. * Fixed uploader names and standard version * Add accounting * Added username@domain to each log message about a reserved session * Corrected pidfile location in init script * Added additional check in report generation for when failure reports are not required * Updated README on certificate/key and configuration file commandline option * Added tls dir to default certificate and key location in sample config * Added another script to just self-sign a certificate/key pair * Automatically load msrp module location in test scripts * Renamed "certs" dir to "tls" * Changed default names of certificates and keys * Removed no longer present CertificateList object from tls.py imports * Updated msrprelay startup script and added option to specify configuration directory and file * Small fix in CA key/cert generation script output * Added much longer expiration time to certificate generation scripts * Modified sample config file and documentation to reflect using SRV records instead of the server name extension * Moved old test scripts to own dir so they do not get included in distribution * Removed file from MANIFEST.in that was no longer present * Modified msrp_send_file.py and msrp_receive_file.py to reflect not using the server name extension, but putting the realm in the To-Path instead * Simplified relay operation and configuration by using the host in the To-Path URI of the AUTH request as the realm during authentication * Fixed memory backend to include domain names * Made key/certificate generation scripts more resilient * Added temporary SIPThor module, just checks credentials in the central database for now * Added uploaders * Use X509Identity from python-gnutls * Changed runtime directory to /var/run * include msrp-send-file and msrp-receive-file scripts in installer * Removed all references to CAs and CRLs, removed certificate checking accordingly, modified config.ini.sample to reflect, put port options together with address * Explicitly mentioned in the sample config file that the default domain needs a Domain config section * Fixed settings name * Update sample config file with a proper description for the relay hostname -- Ruud Klaver Mon, 21 Jan 2008 19:16:38 +0100 diff --git a/debian/control b/debian/control index 0e9ae4c..9ecf0c8 100644 --- a/debian/control +++ b/debian/control @@ -1,22 +1,21 @@ Source: msrprelay Section: net Priority: optional -Maintainer: Dan Pascu +Maintainer: Adrian Georgescu Uploaders: Tijmen de Mes -Build-Depends: debhelper (>= 11), dh-python, python +Build-Depends: debhelper (>= 11), dh-python, python3 Standards-Version: 4.5.0 Package: msrprelay Architecture: all -Depends: ${python:Depends}, ${misc:Depends}, - python-application (>= 2.8.0), - python-gnutls (>= 3.0.0), - python-twisted-core, - python-twisted-names, - python-sqlobject, - python-systemd +Depends: ${python3:Depends}, ${misc:Depends}, + python3-application, + python3-gnutls, + python3-twisted, + python3-sqlobject, + python3-systemd Description: MSRP Relay, a RFC 4976 compatible IM/File transfer relay This software implements an MSRP relay, which is an extension to the MSRP protocol (RFC 4975). Its main role is to help NAT traversal of Interactive Messaging and file transfer sessions for SIP/MSRP endpoints located behind NAT. diff --git a/debian/copyright b/debian/copyright index e4faccd..0a0d951 100644 --- a/debian/copyright +++ b/debian/copyright @@ -1,22 +1,22 @@ -Copyright 2008-2020 AG Projects +Copyright 2008-2021 AG Projects http://ag-projects.com License This package is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2, as published by the Free Software Foundation. This package is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this package; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA On Debian systems, the complete text of the GNU General Public License can be found in `/usr/share/common-licenses/GPL-2'. diff --git a/debian/rules b/debian/rules index 098c539..c66cac0 100755 --- a/debian/rules +++ b/debian/rules @@ -1,20 +1,20 @@ #!/usr/bin/make -f %: - dh $@ --with python2 --buildsystem=pybuild + dh $@ --with python3 --buildsystem=pybuild override_dh_clean: dh_clean rm -rf build dist MANIFEST override_dh_auto_install: dh_auto_install mv debian/msrprelay/etc/msrprelay/config.ini.sample debian/msrprelay/etc/msrprelay/config.ini override_dh_installsystemd: dh_installsystemd --no-start override_dh_fixperms: dh_fixperms chmod 600 debian/msrprelay/etc/msrprelay/config.ini diff --git a/makedeb.sh b/makedeb.sh new file mode 100755 index 0000000..2e098a8 --- /dev/null +++ b/makedeb.sh @@ -0,0 +1,13 @@ +#!/bin/bash +if [ -f dist ]; then + rm -r dist +fi + +python3 setup.py sdist +cd dist + +tar zxvf *.tar.gz +cd msrprelay-?.?.? + +debuild --no-sign + diff --git a/msrp/__init__.py b/msrp/__init__.py index 9e92fa0..3eab90f 100644 --- a/msrp/__init__.py +++ b/msrp/__init__.py @@ -1,4 +1,4 @@ -__version__ = '1.3.1' +__version__ = '1.4.0' configuration_file = 'config.ini' diff --git a/msrp/backend/sipthor.py b/msrp/backend/sipthor.py index 21bd26c..934b060 100644 --- a/msrp/backend/sipthor.py +++ b/msrp/backend/sipthor.py @@ -1,129 +1,137 @@ -import cjson +import json import signal from application import log from application.configuration import ConfigSection, ConfigSetting from application.python.types import Singleton from application.system import host from application.process import process from gnutls.interfaces.twisted import TLSContext, X509Credentials from sqlobject import sqlhub, connectionForURI, SQLObject, StringCol, BLOBCol from sqlobject.dberrors import Error as SQLObjectError from twisted.internet.threads import deferToThread from msrp import configuration_file, __version__ from msrp.digest import LoginFailed from msrp.tls import Certificate, PrivateKey from thor.eventservice import EventServiceClient, ThorEvent from thor.entities import ThorEntitiesRoleMap, GenericThorEntity as ThorEntity from thor.tls import X509NameValidator +class ThorEntityAddress(bytes): + def __new__(cls, ip, control_port=None, version='unknown'): + instance = super().__new__(cls, ip.encode('utf-8')) + instance.ip = ip + instance.version = version + instance.control_port = control_port + return instance + + class Config(ConfigSection): __cfgfile__ = configuration_file __section__ = 'SIPThor' cleartext_passwords = True uri = "mysql://user:pass@db/sipthor" subscriber_table = "sip_accounts" username_col = "username" domain_col = "domain" profile_col = "profile" multiply = 1000 certificate = ConfigSetting(type=Certificate, value=None) private_key = ConfigSetting(type=PrivateKey, value=None) ca = ConfigSetting(type=Certificate, value=None) class ThorNetworkConfig(ConfigSection): __cfgfile__ = configuration_file __section__ = 'ThorNetwork' domain = "sipthor-domain" passport = X509NameValidator('O:undefined, OU:undefined') class Subscribers(SQLObject): class sqlmeta: table = Config.subscriber_table username = StringCol(dbName=Config.username_col) domain = StringCol(dbName=Config.domain_col) profile = BLOBCol(dbName=Config.profile_col) sqlhub.processConnection = connectionForURI(Config.uri) -class ThorNetworkService(EventServiceClient): - __metaclass__ = Singleton +class ThorNetworkService(EventServiceClient, metaclass=Singleton): topics = ["Thor.Members"] def __init__(self): self.node = ThorEntity(host.default_ip, ['msrprelay_server'], version=__version__) self.networks = {} self.presence_message = ThorEvent('Thor.Presence', self.node.id) self.shutdown_message = ThorEvent('Thor.Leave', self.node.id) credentials = X509Credentials(Config.certificate, Config.private_key, [Config.ca]) credentials.verify_peer = True tls_context = TLSContext(credentials) EventServiceClient.__init__(self, ThorNetworkConfig.domain, tls_context) process.signals.add_handler(signal.SIGHUP, self._handle_signal) process.signals.add_handler(signal.SIGINT, self._handle_signal) process.signals.add_handler(signal.SIGTERM, self._handle_signal) def handle_event(self, event): # print "Received event: %s" % event networks = self.networks role_map = ThorEntitiesRoleMap(event.message) # mapping between role names and lists of nodes with that role for role in ["msrprelay_server"]: try: network = networks[role] # avoid setdefault here because it always evaluates the 2nd argument except KeyError: from thor import network as thor_network network = thor_network.new(Config.multiply) networks[role] = network - new_nodes = set([node.ip for node in role_map.get(role, [])]) + new_nodes = set([ThorEntityAddress(node.ip, getattr(node, 'control_port', None), getattr(node, 'version', 'unknown')) for node in role_map.get(role, [])]) + old_nodes = set(network.nodes) added_nodes = new_nodes - old_nodes removed_nodes = old_nodes - new_nodes if removed_nodes: for node in removed_nodes: network.remove_node(node) plural = len(removed_nodes) != 1 and 's' or '' - log.info('removed %s node%s: %s', role, plural, ', '.join(removed_nodes)) + log.info("removed %s node%s: %s" % (role, plural, ', '.join([node.decode() for node in removed_nodes]))) if added_nodes: for node in added_nodes: network.add_node(node) plural = len(added_nodes) != 1 and 's' or '' - log.info('added %s node%s: %s', role, plural, ', '.join(added_nodes)) - # print "Thor %s nodes: %s" % (role, str(network.nodes)) + log.info('added %s node%s: %s' % (role, plural, ', '.join([node.decode() for node in added_nodes]))) class Checker(object): def __init__(self): self.cleartext_passwords = Config.cleartext_passwords self._thor_service = ThorNetworkService() def _retrieve(self, col, username, domain): try: subscriber = Subscribers.selectBy(username=username, domain=domain)[0] except IndexError: raise LoginFailed("Username not found") except SQLObjectError: raise LoginFailed("Database error") try: - profile = cjson.decode(subscriber.profile) - except cjson.DecodeError: + profile = json.loads(subscriber.profile) + except ValueError: raise LoginFailed("Database JSON error") try: return profile[col] except KeyError: raise LoginFailed("Database profile error") def retrieve_password(self, username, domain): return deferToThread(self._retrieve, "password", username, domain) def retrieve_ha1(self, username, domain): return deferToThread(self._retrieve, "ha1", username, domain) diff --git a/msrp/digest.py b/msrp/digest.py index 8e96ad1..4095eac 100644 --- a/msrp/digest.py +++ b/msrp/digest.py @@ -1,110 +1,113 @@ from base64 import b64encode, b64decode from hashlib import md5 -from os import urandom from time import time +import random + +def get_random_data(length): + return ''.join(chr(random.randint(0, 255)) for x in range(length)) class LoginFailed(Exception): pass def calc_ha1(**parameters): ha1_text = "%(username)s:%(realm)s:%(password)s" % parameters - return md5(ha1_text).hexdigest() + return md5(ha1_text.encode('utf-8')).hexdigest() def calc_ha2_response(**parameters): ha2_text = "%(method)s:%(uri)s" % parameters - return md5(ha2_text).hexdigest() + return md5(ha2_text.encode('utf-8')).hexdigest() def calc_ha2_rspauth(**parameters): ha2_text = ":%(uri)s" % parameters - return md5(ha2_text).hexdigest() + return md5(ha2_text.encode('utf-8')).hexdigest() def calc_hash(**parameters): hash_text = "%(ha1)s:%(nonce)s:%(nc)s:%(cnonce)s:auth:%(ha2)s" % parameters - return md5(hash_text).hexdigest() + return md5(hash_text.encode('utf-8')).hexdigest() def calc_responses(**parameters): - if parameters.has_key("ha1"): + if "ha1" in parameters: ha1 = parameters.pop("ha1") else: ha1 = calc_ha1(**parameters) ha2_response = calc_ha2_response(**parameters) ha2_rspauth = calc_ha2_rspauth(**parameters) response = calc_hash(ha1 = ha1, ha2 = ha2_response, **parameters) rspauth = calc_hash(ha1 = ha1, ha2 = ha2_rspauth, **parameters) return response, rspauth def process_www_authenticate(username, password, method, uri, **parameters): nc = "00000001" - cnonce = urandom(16).encode("hex") + cnonce = get_random_data(16).encode().hex() parameters["username"] = username parameters["password"] = password parameters["method"] = method parameters["uri"] = uri response, rsp_auth = calc_responses(nc = nc, cnonce = cnonce, **parameters) authorization = {} authorization["username"] = username authorization["realm"] = parameters["realm"] authorization["nonce"] = parameters["nonce"] authorization["qop"] = "auth" authorization["nc"] = nc authorization["cnonce"] = cnonce authorization["response"] = response authorization["opaque"] = parameters["opaque"] return authorization, rsp_auth class AuthChallenger(object): def __init__(self, expire_time): self.expire_time = expire_time - self.key = urandom(16) + self.key = get_random_data(16) def generate_www_authenticate(self, realm, peer_ip): www_authenticate = {} www_authenticate["realm"] = realm www_authenticate["qop"] = "auth" - nonce = urandom(16) + "%.3f:%s" % (time(), peer_ip) - www_authenticate["nonce"] = b64encode(nonce) - opaque = md5(nonce + self.key) + nonce = get_random_data(16) + "%.3f:%s" % (time(), peer_ip) + www_authenticate["nonce"] = b64encode(nonce.encode()).decode() + opaque = md5((nonce + self.key).encode()) www_authenticate["opaque"] = opaque.hexdigest() return www_authenticate def process_authorization_ha1(self, ha1, method, uri, peer_ip, **parameters): parameters["method"] = method parameters["uri"] = uri try: nonce = parameters["nonce"] opaque = parameters["opaque"] response = parameters["response"] - except IndexError, e: + except IndexError as e: raise LoginFailed("Parameter not present: %s", e.message) try: expected_response, rspauth = calc_responses(ha1 = ha1, **parameters) except: raise #raise LoginFailed("Parameters error") if response != expected_response: raise LoginFailed("Incorrect password") try: - nonce_dec = b64decode(nonce) + nonce_dec = b64decode(nonce.encode()).decode() issued, nonce_ip = nonce_dec[16:].split(":", 1) issued = float(issued) except: raise LoginFailed("Could not decode nonce") if nonce_ip != peer_ip: raise LoginFailed("This challenge was not issued to you") - expected_opaque = md5(nonce_dec + self.key).hexdigest() + expected_opaque = md5((nonce_dec + self.key).encode()).hexdigest() if opaque != expected_opaque: raise LoginFailed("This nonce/opaque combination was not issued by me") if issued + self.expire_time < time(): raise LoginFailed("This challenge has expired") authentication_info = {} authentication_info["qop"] = "auth" authentication_info["cnonce"] = parameters["cnonce"] authentication_info["nc"] = parameters["nc"] authentication_info["rspauth"] = rspauth return authentication_info def process_authorization_password(self, password, method, uri, peer_ip, **parameters): ha1 = calc_ha1(password = password, **parameters) return self.process_authorization_ha1(ha1, method, uri, peer_ip, **parameters) diff --git a/msrp/protocol.py b/msrp/protocol.py index a0f7c4f..a7f15d9 100644 --- a/msrp/protocol.py +++ b/msrp/protocol.py @@ -1,531 +1,535 @@ import re from collections import deque from random import getrandbits from twisted.protocols.basic import LineReceiver class MSRPError(Exception): pass class ParsingError(MSRPError): pass class HeaderParsingError(ParsingError): def __init__(self, header): self.header = header ParsingError.__init__(self, "Error parsing %s header" % header) class MSRPHeaderMeta(type): header_classes = {} def __init__(cls, name, bases, dict): type.__init__(cls, name, bases, dict) try: cls.header_classes[dict["name"]] = name except KeyError: pass def _auth_header_quote(name, value, header_name): if name == "qop": if header_name == "WWW-Authenticate": return '"%s"' % value else: return value if name in ("stale", "algorithm", "nc"): return value else: return '"%s"' % value -class MSRPHeader(object): - __metaclass__ = MSRPHeaderMeta - +class MSRPHeader(object, metaclass=MSRPHeaderMeta): def __new__(cls, name, value): if isinstance(value, str) and name in MSRPHeaderMeta.header_classes: cls = eval(MSRPHeaderMeta.header_classes[name]) return object.__new__(cls) def __init__(self, name, value): self.name = name if isinstance(value, str): self.encoded = value else: self.decoded = value def _raise_error(self): raise HeaderParsingError(self.name) def _get_encoded(self): if self._encoded is None: self._encoded = self._encode(self._decoded) return self._encoded def _set_encoded(self, encoded): self._encoded = encoded self._decoded = None encoded = property(_get_encoded, _set_encoded) def _get_decoded(self): if self._decoded is None: self._decoded = self._decode(self._encoded) return self._decoded def _set_decoded(self, decoded): self._decoded = decoded self._encoded = None decoded = property(_get_decoded, _set_decoded) def _decode(self, encoded): return encoded def _encode(self, decoded): return decoded def clone(self): return self.__class__(self.name, self.encoded) class MSRPNamedHeader(MSRPHeader): def __new__(cls, *args): if len(args) == 1: value = args[0] else: value = args[1] return MSRPHeader.__new__(cls, cls.name, value) def __init__(self, *args): if len(args) == 1: value = args[0] else: value = args[1] MSRPHeader.__init__(self, self.name, value) class URIHeader(MSRPNamedHeader): def _decode(self, encoded): try: return deque(parse_uri(uri) for uri in encoded.split(" ")) except ParsingError: self._raise_error() def _encode(self, decoded): return " ".join([str(uri) for uri in decoded]) class IntegerHeader(MSRPNamedHeader): def _decode(self, encoded): try: return int(encoded) except ValueError: self._raise_error() def _encode(self, decoded): return str(decoded) class DigestHeader(MSRPNamedHeader): def _decode(self, encoded): try: algo, params = encoded.split(" ", 1) except ValueError: self._raise_error() if algo != "Digest": self._raise_error() try: param_dict = dict((x.strip('"') for x in param.split("=", 1)) for param in params.split(", ")) except: self._raise_error() return param_dict def _encode(self, decoded): - return "Digest " + ", ".join(['%s=%s' % (name, _auth_header_quote(name, value, self.name)) for name, value in decoded.iteritems()]) + return "Digest " + ", ".join(['%s=%s' % (name, _auth_header_quote(name, value, self.name)) for name, value in decoded.items()]) class ToPathHeader(URIHeader): name = "To-Path" class FromPathHeader(URIHeader): name = "From-Path" class MessageIDHeader(MSRPNamedHeader): name = "Message-ID" class SuccessReportHeader(MSRPNamedHeader): name = "Success-Report" def _decode(self, encoded): if encoded not in ["yes", "no"]: self._raise_error() return encoded class FailureReportHeader(MSRPNamedHeader): name = "Failure-Report" def _decode(self, encoded): if encoded not in ["yes", "no", "partial"]: self._raise_error() return encoded class ByteRangeHeader(MSRPNamedHeader): name = "Byte-Range" def _decode(self, encoded): try: rest, total = encoded.split("/") fro, to = rest.split("-") fro = int(fro) except ValueError: self._raise_error() try: to = int(to) except ValueError: if to != "*": self._raise_error() to = None try: total = int(total) except ValueError: if total != "*": self._raise_error() total = None return [fro, to, total] def _encode(self, decoded): fro, to, total = decoded if to is None: to = "*" if total is None: total = "*" return "%s-%s/%s" % (fro, to, total) class StatusHeader(MSRPNamedHeader): name = "Status" def _decode(self, encoded): try: namespace, rest = encoded.split(" ", 1) except ValueError: self._raise_error() if namespace != "000": self._raise_error() rest_sp = rest.split(" ", 1) try: if len(rest_sp[0]) != 3: raise ValueError code = int(rest_sp[0]) except ValueError: self._raise_error() try: comment = rest_sp[1] except IndexError: comment = None return code, comment def _encode(self, decoded): code, comment = decoded encoded = "000 %03d" % code if comment is not None: encoded += " %s" % comment return encoded class ExpiresHeader(IntegerHeader): name = "Expires" class MinExpiresHeader(IntegerHeader): name = "Min-Expires" class MaxExpiresHeader(IntegerHeader): name = "Max-Expires" class UsePathHeader(URIHeader): name = "Use-Path" class WWWAuthenticateHeader(DigestHeader): name = "WWW-Authenticate" class AuthorizationHeader(DigestHeader): name = "Authorization" class AuthenticationInfoHeader(MSRPNamedHeader): name = "Authentication-Info" def _decode(self, encoded): try: param_dict = dict((x.strip('"') for x in param.split("=", 1)) for param in encoded.split(", ")) except: self._raise_error() return param_dict def _encode(self, decoded): - return ", ".join(['%s=%s' % (name, _auth_header_quote(name, value, self.name)) for name, value in decoded.iteritems()]) + return ", ".join(['%s=%s' % (name, _auth_header_quote(name, value, self.name)) for name, value in decoded.items()]) class ContentTypeHeader(MSRPNamedHeader): name = "Content-Type" class ContentIDHeader(MSRPNamedHeader): name = "Content-ID" class ContentDescriptionHeader(MSRPNamedHeader): name = "Content-Description" class ContentDispositionHeader(MSRPNamedHeader): name = "Content-Disposition" def _decode(self, encoded): try: sp = encoded.split(";") disposition = sp[0] parameters = dict(param.split("=", 1) for param in sp[1:]) except: self._raise_error() return [disposition, parameters] def _encode(self, decoded): disposition, parameters = decoded - return ";".join([disposition] + ["%s=%s" % pair for pair in parameters.iteritems()]) + return ";".join([disposition] + ["%s=%s" % pair for pair in parameters.items()]) class MSRPData(object): def __init__(self, transaction_id, method=None, code=None, comment=None, data=''): self.transaction_id = transaction_id self.method = method self.code = code self.comment = comment self.data = data self.headers = {} def __str__(self): if self.method is None: description = "MSRP response: %03d" % self.code if self.comment is not None: description += " %s" % self.comment else: description = "MSRP %s request" % self.method return description def add_header(self, header): self.headers[header.name] = header def verify_headers(self): try: # Decode To-/From-path headers first to be able to send responses self.headers["To-Path"].decoded self.headers["From-Path"].decoded - except KeyError, e: + except KeyError as e: raise HeaderParsingError(e.args[0]) - for header in self.headers.itervalues(): + for header in self.headers.values(): header.decoded @property def failure_report(self): if "Failure-Report" in self.headers: return self.headers["Failure-Report"].decoded else: return "yes" @property def success_report(self): if "Success-Report" in self.headers: return self.headers["Success-Report"].decoded else: return "no" def encode_start(self): data = [] if self.method is not None: data.append("MSRP %(transaction_id)s %(method)s" % self.__dict__) else: data.append("MSRP %(transaction_id)s %(code)03d" % self.__dict__ + (self.comment is not None and " %s" % self.comment or "")) headers = self.headers.copy() data.append("To-Path: %s" % headers.pop("To-Path").encoded) data.append("From-Path: %s" % headers.pop("From-Path").encoded) - for hnameval in [(hname, headers.pop(hname).encoded) for hname in headers.keys() if not hname.startswith("Content-")]: + for hnameval in [(hname, headers.pop(hname).encoded) for hname in list(headers.keys()) if not hname.startswith("Content-")]: data.append("%s: %s" % hnameval) - for hnameval in [(hname, headers.pop(hname).encoded) for hname in headers.keys() if hname != "Content-Type"]: + for hnameval in [(hname, headers.pop(hname).encoded) for hname in list(headers.keys()) if hname != "Content-Type"]: data.append("%s: %s" % hnameval) if len(headers) > 0: data.append("Content-Type: %s" % headers["Content-Type"].encoded) data.append("") data.append("") return "\r\n".join(data) def encode_end(self, continuation): return "\r\n-------%s%s\r\n" % (self.transaction_id, continuation) def encode(self, continuation='$'): return self.encode_start() + self.data + self.encode_end(continuation) def clone(self): other = self.__class__(self.transaction_id, self.method, self.code, self.comment, self.data) - for header in self.headers.itervalues(): + for header in self.headers.values(): other.add_header(header.clone()) return other class MSRPProtocol(LineReceiver): MAX_LENGTH = 16384 MAX_LINES = 64 def __init__(self): self.peer = None self._reset() def _reset(self): self.data = None self.line_count = 0 def connectionMade(self): self.peer = self.factory.get_peer(self) def lineReceived(self, line): + try: + decoded_line = line.decode('utf-8') + except UnicodeDecodeError: + decoded_line = None + if self.data: if len(line) == 0: self.term_buf_len = 12 + len(self.data.transaction_id) - self.term_buf = "" + self.term_buf = b"" self.term = re.compile("^(.*)\r\n-------%s([$#+])\r\n(.*)$" % re.escape(self.data.transaction_id), re.DOTALL) self.peer.data_start(self.data) self.setRawMode() else: - match = self.term.match(line) + match = self.term.match(decoded_line) if decoded_line else None if match: continuation = match.group(1) self.peer.data_start(self.data) self.peer.data_end(continuation) self._reset() else: self.line_count += 1 if self.line_count > self.MAX_LINES: self._reset() return try: - hname, hval = line.split(": ", 2) + name, value = decoded_line.split(": ", 2) except ValueError: return # let this pass silently, we'll just not read this line else: - self.data.add_header(MSRPHeader(hname, hval)) + self.data.add_header(MSRPHeader(name, value)) else: # we received a new message try: - msrp, transaction_id, rest = line.split(" ", 2) + msrp, transaction_id, rest = decoded_line.split(" ", 2) except ValueError: return # drop connection? if msrp != "MSRP": return # drop connection? method, code, comment = None, None, None rest_sp = rest.split(" ", 1) try: if len(rest_sp[0]) != 3: raise ValueError code = int(rest_sp[0]) except ValueError: # we have a request method = rest_sp[0] else: # we have a response if len(rest_sp) > 1: comment = rest_sp[1] self.data = MSRPData(transaction_id, method, code, comment) self.term = re.compile("^-------%s([$#+])$" % re.escape(transaction_id)) def lineLengthExceeded(self, line): self._reset() def rawDataReceived(self, data): match_data = self.term_buf + data - match = self.term.match(match_data) + match = self.term.match(match_data.decode()) + if match: # we got the last data for this message contents, continuation, extra = match.groups() contents = contents[len(self.term_buf):] if contents: self.peer.write_chunk(contents) self.peer.data_end(continuation) self._reset() self.setLineMode(extra) else: self.peer.write_chunk(data) self.term_buf = match_data[-self.term_buf_len:] def connectionLost(self, reason): if self.peer: self.peer.connection_lost(reason.value) _re_uri = re.compile("^(?P.*?)://(((?P.*?)@)?(?P.*?)(:(?P[0-9]+?))?)(/(?P.*?))?;(?P.*?)(;(?P.*))?$") def parse_uri(uri_str): match = _re_uri.match(uri_str) if match is None: raise ParsingError("Cannot parse URI") uri_params = match.groupdict() if uri_params["port"] is not None: uri_params["port"] = int(uri_params["port"]) if uri_params["parameters"] is not None: try: uri_params["parameters"] = dict(param.split("=") for param in uri_params["parameters"].split(";")) except ValueError: raise ParsingError("Cannot parse URI parameters") scheme = uri_params.pop("scheme") if scheme == "msrp": uri_params["use_tls"] = False elif scheme == "msrps": uri_params["use_tls"] = True else: raise ParsingError("Invalid scheme user in URI: %s" % scheme) if uri_params["transport"] != "tcp": raise ParsingError('Invalid transport in URI, only "tcp" is accepted: %s' % uri_params["transport"]) return URI(**uri_params) class URI(object): def __init__(self, host, use_tls = False, user = None, port = None, session_id = None, transport = "tcp", parameters = None): self.use_tls = use_tls self.user = user self.host = host self.port = port self.session_id = session_id self.transport = transport if parameters is None: self.parameters = {} else: self.parameters = parameters def __str__(self): uri_str = [] if self.use_tls: uri_str.append("msrps://") else: uri_str.append("msrp://") if self.user: uri_str.extend([self.user, "@"]) uri_str.append(self.host) if self.port: uri_str.extend([":", str(self.port)]) if self.session_id: uri_str.extend(["/", self.session_id]) uri_str.extend([";", self.transport]) - for key, value in self.parameters.iteritems(): + for key, value in self.parameters.items(): uri_str.extend([";", key, "=", value]) return "".join(uri_str) def __eq__(self, other): """MSRP URI comparison according to section 6.1 of RFC 4975""" if self is other: return True if self.use_tls != other.use_tls: return False if self.host.lower() != other.host.lower(): return False if self.port != other.port: return False if self.session_id != other.session_id: return False if self.transport.lower() != other.transport.lower(): return False return True def __ne__(self, other): return not self == other def generate_transaction_id(): return '%024x' % getrandbits(96) diff --git a/msrp/relay.py b/msrp/relay.py index 8be6c25..2fb4e31 100644 --- a/msrp/relay.py +++ b/msrp/relay.py @@ -1,784 +1,783 @@ import weakref from copy import copy from collections import deque from application import log from application.configuration import * from application.configuration.datatypes import NetworkAddress, LogLevel from application.python.types import Singleton from application.system import host -from zope.interface import implements +from zope.interface import implementer from twisted.internet.defer import maybeDeferred from twisted.internet import reactor from twisted.internet.protocol import Factory, ClientFactory from twisted.internet.interfaces import IPullProducer from gnutls.interfaces.twisted import TLSContext, X509Credentials from msrp.tls import Certificate, PrivateKey from msrp.protocol import * from msrp.digest import AuthChallenger, LoginFailed from msrp.responses import * from msrp import configuration_file class RelayConfig(ConfigSection): __cfgfile__ = configuration_file __section__ = 'Relay' address = ConfigSetting(type=NetworkAddress, value=NetworkAddress("0.0.0.0:2855")) hostname = "" default_domain = "" allow_other_methods = True session_expiration_time_minimum = 60 session_expiration_time_default = 600 session_expiration_time_maximum = 3600 auth_challenge_expiration_time = 15 backend = "database" max_auth_attempts = 3 debug_notls = False log_failed_auth = False certificate = ConfigSetting(type=Certificate, value=None) key = ConfigSetting(type=PrivateKey, value=None) log_level = ConfigSetting(type=LogLevel, value=log.level.DEBUG) -class Relay(object): - __metaclass__ = Singleton - +class Relay(object, metaclass=Singleton): def __init__(self): self.unbound_sessions = {} self._do_init() def _do_init(self): self.listener = None self.backend = __import__("msrp.backend.%s" % RelayConfig.backend.lower(), globals(), locals(), [""]).Checker() if not RelayConfig.debug_notls: if RelayConfig.certificate is None: raise RuntimeError("TLS certificate file is not specified in configuration or is invalid") if RelayConfig.key is None: raise RuntimeError("TLS private key file is not specified in configuration or is invalid") self.credentials = X509Credentials(RelayConfig.certificate, RelayConfig.key) self.credentials.verify_peer = False # TODO: add configuration option for configuring session parameters? -Saul if RelayConfig.hostname != "": self.hostname = RelayConfig.hostname if not RelayConfig.debug_notls: def matches(hostname, pattern): if pattern.startswith('*.'): return hostname.endswith(pattern[1:]) else: return hostname == pattern - if not any(matches(self.hostname, name) for name in RelayConfig.certificate.alternative_names.dns): + if not any(matches(self.hostname, name.decode()) for name in RelayConfig.certificate.alternative_names.dns): raise RuntimeError('The specified MSRP Relay hostname "%s" is not set as DNS subject alternative name in the TLS certificate.' % self.hostname) elif not RelayConfig.debug_notls: self.hostname = RelayConfig.certificate.alternative_names.dns[0] # Just grab the first one? elif RelayConfig.address[0] != "0.0.0.0": self.hostname = RelayConfig.address[0] else: self.hostname = host.default_ip self.auth_challenger = AuthChallenger(RelayConfig.auth_challenge_expiration_time) def _do_run(self): if RelayConfig.debug_notls: self.listener = reactor.listenTCP(RelayConfig.address[1], RelayFactory(), interface=RelayConfig.address[0]) else: self.listener = reactor.listenTLS(RelayConfig.address[1], RelayFactory(), TLSContext(self.credentials), interface=RelayConfig.address[0]) def run(self): self._do_run() reactor.run() def reload(self): log.debug('Reloading configuration file') RelayConfig.reset() RelayConfig.read() if not self.listener: try: self._do_init() - except RuntimeError, e: + except RuntimeError as e: log.critical('Error reloading configuration file: %s' % e) reactor.stop() else: result = self.listener.stopListening() result.addCallback(lambda x: self._do_init()) result.addCallbacks(lambda x: self._do_run(), self._reload_failure) def _reload_failure(self, failure): failure.trap(RuntimeError) log.critical('Error reloading configuration file: %s' % failure.value) reactor.stop() def generate_uri(self): return URI(self.hostname, port=RelayConfig.address[1], use_tls=not RelayConfig.debug_notls) class RelayFactory(Factory): protocol = MSRPProtocol noisy = False def get_peer(self, protocol): peer = Peer(protocol = protocol) return peer class ConnectingFactory(ClientFactory): protocol = MSRPProtocol noisy = False def __init__(self, peer): self.peer = peer def get_peer(self, protocol): self.peer.got_protocol(protocol) return self.peer def clientConnectionFailed(self, connector, reason): self.peer.connection_failed(reason.value) class ForwardingData(object): def __init__(self, msrpdata): self.msrpdata_received = msrpdata self.msrpdata_forward = None self.bytes_received = 0 self.data_queue = deque() self.continuation = None def __str__(self): return str(self.msrpdata_received) @property def method(self): return self.msrpdata_received.method @property def bytes_in_queue(self): return sum(len(data) for data in self.data_queue) def add_data(self, data): self.bytes_received += len(data) self.data_queue.append(data) def consume_data(self): if self.data_queue: return self.data_queue.popleft() else: return None class ForwardingSendData(ForwardingData): def __init__(self, msrpdata): super(ForwardingSendData, self).__init__(msrpdata) self.position = msrpdata.headers["Byte-Range"].decoded[0] class PeerLogger(log.ContextualLogger): def __init__(self, peer): super(PeerLogger, self).__init__(logger=log.get_logger()) self.peer = weakref.ref(peer) def apply_context(self, message): peer = self.peer() if message == '' or peer is None: return message # cannot apply context if there is no message or peer to provide the context elif peer.session is not None: return 'session {session.session_id} for {session.username}@{session.realm} ({state}): {message}'.format(session=peer.session, state=peer.state, message=message) elif peer.protocol is not None: return '{address.host}:{address.port} ({state}): {message}'.format(address=peer.protocol.transport.getPeer(), state=peer.state, message=message) else: return message # peer exists, but both peer.session and peer.protocol are None so there is no context to apply # A session always exists of two peer instances that are associated with # eachother. # # A Peer instance has an attribute state, which can be one of the following: # # NEW: # A newly created incoming connection. If an AUTH is received and successfully # processed this creates a new session and moves the Peer into the UNBOUND # state. Alternatively, a SEND for an existing session can be received, which # directly binds the Peer with the session and moves it into ESTABLISHED. # # UNBOUND: # A newly created session which does not yet have another endpoint associated # with it. This association can occur either through a new outgoing connection # when the client that performed the AUTH sends a SEND message or when a new # SEND message is received. # # CONNECTING: # An attempt at a new outgoing connection. Messages can already be queued on it. # When the protocol is connected the Peer will move into ESTABLISHED. # # ESTABLISHED: # The two endpoints for the session are known and messages can be passed through # in either direction. When the connection to this peer is lost, the Peer will # move into DISCONNECTED, when the connection to the other peer is lost it will # move into DISCONNECTING. # # DISCONNECTING: # The connection to the other peer is lost and relay has to send REPORTs for # queued messages that require it. After this it will disconnect itself. # # DISCONNECTED: # When the peer is disconnected, which can happen at any time, inform the other # associated peer. This is always the end state. # # | | # V V # +-----+ +------------+ # +-----| NEW |---------+ +--------| CONNECTING |-----+ # | +-----+ | | +------------+ | # | | | | | # | V V V | # | +---------+ +-------------+ +---------------+ | # |<--| UNBOUND |-->| ESTABLISHED |-->| DISCONNECTING |-->| # | +---------+ +-------------+ +---------------+ | # | | | # | V | # | +--------------+ | # +---------------->| DISCONNECTED |<---------------------+ # +--------------+ +@implementer(IPullProducer) class Peer(object): - implements(IPullProducer) def __init__(self, session = None, protocol = None, path = None, other_peer = None): self.session = session self.protocol = protocol self.other_peer = other_peer self.path = path if self.protocol is not None: self.state = "NEW" self.auth_attempts = 0 self.invalid_timer = reactor.callLater(30, self._cb_invalid) else: self.invalid_timer = None self.state = "CONNECTING" self.send_transactions = {} self.other_transactions = {} # Other requests not REPORT or SEND self.forwarding_data = None # transmission attributes self.registered = False self.forwarding_send_request = False self.forward_send_queue = deque() self.forward_other_queue = deque() self.read_paused = False self.relay = Relay() self.logger = PeerLogger(self) # called by MSRPProtocol def data_start(self, msrpdata): # self.logger.debug('Received headers for %s', msrpdata) if self.state == "NEW": result = maybeDeferred(self._unbound_peer_data, msrpdata) result.addErrback(self._cb_catch_response, msrpdata) else: self._bound_peer_data(msrpdata) def write_chunk(self, chunk): # self.logger.debug("Received %d bytes of MSRP body", len(chunk)) if self.state == "ESTABLISHED" and self.forwarding_data: self.forwarding_data.add_data(chunk) if self.forwarding_data.method != 'SEND' and self.forwarding_data.bytes_in_queue > 10240: self.logger.debug('Non-SEND request payload bigger than 10KB, closing connection') del self.other_transactions[self.forwarding_data.msrpdata_forward.transaction_id] self.forwarding_data = None self.protocol.transport.loseConnection() return self.other_peer._maybe_pause_read() self.other_peer.start_sending() def data_end(self, continuation): # self.logger.debug('Received termination "%s"', continuation) if self.state == "ESTABLISHED" and self.forwarding_data: msrpdata = self.forwarding_data.msrpdata_received if msrpdata.method == "SEND": if msrpdata.failure_report != "no": if msrpdata.failure_report == "yes": self.enqueue(ResponseOK(msrpdata).data.encode()) self.send_transactions[self.forwarding_data.msrpdata_forward.transaction_id] = (self.forwarding_data, None) self.forwarding_data.continuation = continuation else: if msrpdata.method != 'REPORT' and msrpdata.failure_report != 'no': # Keep track, for matching replies timer = reactor.callLater(30, self._cb_transaction_timeout, self.forwarding_data) self.other_transactions[self.forwarding_data.msrpdata_forward.transaction_id] = (self.forwarding_data, timer) # For methods other than SEND, assemble the chunk and don't use ForwardingData self.forwarding_data.msrpdata_forward.data = ''.join(self.forwarding_data.data_queue) self.other_peer.enqueue(self.forwarding_data.msrpdata_forward.encode(continuation)) self.other_peer.start_sending() self.forwarding_data = None def connection_lost(self, reason): - self.logger.debug('Connection lost: %s', reason) + self.logger.debug('Connection with %s lost: %s' % (self.other_peer, reason)) if self.invalid_timer and self.invalid_timer.active(): self.invalid_timer.cancel() self.invalid_timer = None if self.state == "ESTABLISHED": self.other_peer.disconnect() if self.state == "UNBOUND": del self.relay.unbound_sessions[self.session.session_id] self.state = "DISCONNECTED" if self.session is not None and self is self.session.source: self.logger.debug('bytes sent: %d, bytes received: %d', self.session.upstream_bytes, self.session.downstream_bytes) self._cleanup() # called by ConnectingFactory def connection_failed(self, reason): - self.logger.warning('Connection failed: %s', reason) + self.logger.warning('Connection with %s failed: %s' % (self.other_peer, reason)) if self.state == "CONNECTING": self.other_peer.disconnect() self._cleanup() # methods for an unbound peer def _cb_invalid(self): self.logger.warning('No valid MSRP message received, disconnecting') self.invalid_timer = None self.disconnect() def _cb_catch_response(self, failure, msrpdata): failure.trap(ResponseException) if msrpdata.method is None: self.logger.warning('Caught exception to response: %s (%s)', failure.value.__class__.__name__, failure.value.data.comment) return response = failure.value.data # self.logger.debug('Sending response %03d (%s)', response.code, response.comment) self.enqueue(response.encode()) def _unbound_peer_data(self, msrpdata): try: msrpdata.verify_headers() - except ParsingError, e: + except ParsingError as e: if isinstance(e, HeaderParsingError) and (e.header == "To-Path" or e.header == "From-Path"): self.logger.warning('Cannot send error response, path headers unreadable') return else: raise ResponseUnintelligible(msrpdata, e.args[0]) # Check if To-Path is really directed to us. to_path = msrpdata.headers["To-Path"].decoded from_path = msrpdata.headers["From-Path"].decoded if msrpdata.method == "AUTH" and len(to_path) == 1: return self._handle_auth(msrpdata) elif msrpdata.method == "SEND" and len(to_path) > 1: session_id = to_path[0].session_id try: session = self.relay.unbound_sessions[session_id] except KeyError: raise ResponseNoSession(msrpdata) self.logger.debug('Found matching unbound session %s', session.session_id) self.state = "ESTABLISHED" self.invalid_timer.cancel() self.invalid_timer = None self.session = session self.path = from_path self.other_peer = self.session.source self.other_peer.got_destination(self) self._bound_peer_data(msrpdata) else: raise ResponseUnknownMethod(msrpdata) def _handle_auth(self, msrpdata): auth_challenger = self.relay.auth_challenger if RelayConfig.default_domain == "": realm = msrpdata.headers["To-Path"].decoded[0].host else: realm = RelayConfig.default_domain - if not msrpdata.headers.has_key("Authorization"): + if "Authorization" not in msrpdata.headers: # If the Authorization header is not present generate challenge data and respond. www_authenticate = auth_challenger.generate_www_authenticate(realm, self.protocol.transport.getPeer().host) raise ResponseUnauthenticated(msrpdata, headers = [WWWAuthenticateHeader(www_authenticate)]) else: authorization = msrpdata.headers["Authorization"].decoded if authorization.get("realm") != realm: raise ResponseUnauthorized(msrpdata, "realm does not match") if authorization.get("qop") != "auth": raise ResponseUnauthorized(msrpdata, "qop != auth") try: username = authorization["username"] session_id = authorization["nonce"] - except KeyError, e: + except KeyError as e: raise ResponseUnauthorized(msrpdata, "%s field not present in Authorization header" % e.args[0]) if self.relay.backend.cleartext_passwords: result = self.relay.backend.retrieve_password(username, realm) func = auth_challenger.process_authorization_password else: result = self.relay.backend.retrieve_ha1(username, realm) func = auth_challenger.process_authorization_ha1 result.addCallback(func, "AUTH", msrpdata.headers["To-Path"].encoded.split()[-1], self.protocol.transport.getPeer().host, **authorization) result.addCallbacks(self._cb_login_success, self._eb_login_failed, callbackArgs=[msrpdata, session_id, username, realm], errbackArgs=[msrpdata]) return result def _eb_login_failed(self, failure, msrpdata): failure.trap(LoginFailed) self.auth_attempts += 1 if RelayConfig.log_failed_auth: try: username = msrpdata.headers["Authorization"].decoded["username"] except IndexError: self.logger.warning('AUTH failed, no username: %s', failure.value.args[0]) else: self.logger.warning('AUTH failed for username "%s": %s', username, failure.value.args[0]) if self.auth_attempts == RelayConfig.max_auth_attempts: self.disconnect() else: raise ResponseUnauthorized(msrpdata, "Login failed: %s" % failure.value.args[0]) def _cb_login_success(self, authentication_info, msrpdata, session_id, username, realm): # Check the Expires header, if present. - if msrpdata.headers.has_key("Expires"): + if "Expires" in msrpdata.headers: expire = msrpdata.headers["Expires"].decoded if expire < RelayConfig.session_expiration_time_minimum: raise ResponseOutOfBounds(msrpdata, headers = [MinExpiresHeader(RelayConfig.session_expiration_time_minimum)]) if expire > RelayConfig.session_expiration_time_maximum: raise ResponseOutOfBounds(msrpdata, headers = [MaxExpiresHeader(RelayConfig.session_expiration_time_maximum)]) else: expire = RelayConfig.session_expiration_time_default # We got a successful AUTH request, so add a new session # and reply with the the Use-Path. self.state = "UNBOUND" self.invalid_timer.cancel() self.invalid_timer = None from_path = msrpdata.headers["From-Path"].decoded self.path = from_path self.relay.unbound_sessions[session_id] = self.session = Session(self, session_id, expire, username, realm) use_path = copy(from_path) use_path.pop() use_path = list(reversed(use_path)) uri = self.relay.generate_uri() uri.session_id = session_id use_path.append(uri) headers = [UsePathHeader(use_path), ExpiresHeader(expire), AuthenticationInfoHeader(authentication_info)] self.logger.debug('AUTH succeeded, creating new session') raise ResponseOK(msrpdata, headers = headers) # methods for a unconnected peer def got_protocol(self, protocol): self.logger.debug('Successfully connected') self.state = "ESTABLISHED" self.protocol = protocol if self.forward_other_queue or self.forward_send_queue: self.start_sending() def got_destination(self, other_peer): self.state = "ESTABLISHED" del self.relay.unbound_sessions[self.session.session_id] self.session.destination = other_peer self.other_peer = other_peer # methods for a bound and connected peer def _bound_peer_data(self, msrpdata): try: try: msrpdata.verify_headers() - except ParsingError, e: + except ParsingError as e: if isinstance(e, HeaderParsingError) and (e.header == "To-Path" or e.header == "From-Path"): self.logger.error('Cannot send error response, path headers unreadable') return else: raise ResponseUnintelligible(msrpdata, e.args[0]) to_path = copy(msrpdata.headers["To-Path"].decoded) from_path = copy(msrpdata.headers["From-Path"].decoded) relay_uri = to_path.popleft() if relay_uri.session_id != self.session.session_id: raise ResponseNoSession(msrpdata, "Wrong session id on relay MSRP URI") if len(to_path) == 0 and msrpdata.method not in (None, "AUTH"): raise ResponseNoSession(msrpdata, "Non-response with me as endpoint, nowhere to relay to") for index, uri in enumerate(from_path): if uri != self.path[index]: raise ResponseNoSession(msrpdata, "From-Path does not match session source") if msrpdata.method == "AUTH": - if msrpdata.headers.has_key("Expires"): + if "Expires" in msrpdata.headers: expire = msrpdata.headers["Expires"].decoded if expire < RelayConfig.session_expiration_time_minimum: raise ResponseOutOfBounds(msrpdata, headers=[MinExpiresHeader(RelayConfig.session_expiration_time_minimum)]) if expire > RelayConfig.session_expiration_time_maximum: raise ResponseOutOfBounds(msrpdata, headers=[MaxExpiresHeader(RelayConfig.session_expiration_time_maximum)]) else: expire = RelayConfig.session_expiration_time_default use_path = from_path use_path.pop() use_path = list(reversed(use_path)) use_path.append(relay_uri) headers = [UsePathHeader(use_path), ExpiresHeader(expire)] self.logger.debug('Received refreshing AUTH') raise ResponseOK(msrpdata, headers=headers) if self.state == "UNBOUND": if msrpdata.method != "SEND": raise ResponseNoSession(msrpdata, "Non-SEND method received on unbound session") self.got_destination(Peer(path = to_path, session = self.session, other_peer = self)) uri = to_path[0] # self.logger.debug('Attempting to connect to %s', uri) factory = ConnectingFactory(self.other_peer) if uri.use_tls: self.other_peer.connector = reactor.connectTLS(uri.host, uri.port, factory, TLSContext(self.relay.credentials)) else: self.other_peer.connector = reactor.connectTCP(uri.host, uri.port, factory) else: for index, uri in enumerate(to_path): if uri != self.other_peer.path[index]: raise ResponseNoSession(msrpdata, "To-Path does not match session destination") if msrpdata.method == "SEND": - if not msrpdata.headers.has_key("Message-ID"): + if "Message-ID" not in msrpdata.headers: raise ResponseUnintelligible(msrpdata, "SEND received without Message-ID") - if not msrpdata.headers.has_key("Byte-Range"): + if "Byte-Range" not in msrpdata.headers: raise ResponseUnintelligible(msrpdata, "SEND received without Byte-Range") if msrpdata.method is None: # we got a response if msrpdata.transaction_id in self.other_peer.send_transactions: # Handle response to SEND request with Failure-Report != no forwarding_data, timer = self.other_peer.send_transactions.pop(msrpdata.transaction_id) if timer is not None: timer.cancel() if msrpdata.code != ResponseOK.code: report = generate_report(msrpdata.code, forwarding_data, reason=msrpdata.comment) self.other_peer.enqueue(report.encode()) elif msrpdata.transaction_id in self.other_peer.other_transactions: forwarding_data, timer = self.other_peer.other_transactions.pop(msrpdata.transaction_id) if timer is not None: timer.cancel() forward = msrpdata forward.transaction_id = forwarding_data.msrpdata_received.transaction_id to_path_header = msrpdata.headers["To-Path"] to_path = to_path_header.decoded from_path_header = msrpdata.headers["From-Path"] from_path = from_path_header.decoded from_path.appendleft(to_path.popleft()) to_path_header.decoded = to_path from_path_header.decoded = from_path self.other_peer.enqueue(forward.encode()) else: self.logger.debug('Received response for untracked request: %s', msrpdata) elif msrpdata.method in ("SEND", "REPORT", "NICKNAME") or RelayConfig.allow_other_methods: # Do the magic of appending the first To-Path URI to the From-Path. to_path = copy(msrpdata.headers["To-Path"].decoded) from_path = copy(msrpdata.headers["From-Path"].decoded) from_path.appendleft(to_path.popleft()) forward = MSRPData(generate_transaction_id(), method=msrpdata.method) - for header in msrpdata.headers.itervalues(): + for header in msrpdata.headers.values(): forward.add_header(MSRPHeader(header.name, header.encoded)) forward.add_header(ToPathHeader(to_path)) forward.add_header(FromPathHeader(from_path)) if msrpdata.method == 'SEND': self.forwarding_data = ForwardingSendData(msrpdata) else: self.forwarding_data = ForwardingData(msrpdata) self.forwarding_data.msrpdata_forward = forward if self.forwarding_data.method == 'SEND': if msrpdata.failure_report != "no": self.send_transactions[self.forwarding_data.msrpdata_forward.transaction_id] = (self.forwarding_data, None) self.other_peer.enqueue(self.forwarding_data) else: self.other_transactions[self.forwarding_data.msrpdata_forward.transaction_id] = (self.forwarding_data, None) else: raise ResponseUnknownMethod(msrpdata) - except ResponseException, e: + except ResponseException as e: if msrpdata.method is None: self.logger.debug('Caught exception to response: %s (%s)', e.__class__.__name__, e.data.comment) return response = e.data self.enqueue(response.encode()) def _cb_transaction_timeout(self, forward_data): transaction_id = forward_data.msrpdata_forward.transaction_id if forward_data.method == 'SEND': del self.send_transactions[transaction_id] if forward_data.msrpdata_received.failure_report == 'yes': report = generate_report(ResponseDownstreamTimeout.code, forward_data) self.enqueue(report.encode()) else: # We don't really need to do anything here, just remove the request from the mapping del self.other_transactions[transaction_id] def enqueue(self, msrpdata): # self.logger.debug('Enqueuing %s', msrpdata) if isinstance(msrpdata, ForwardingData): self.forward_send_queue.append(msrpdata) else: # a string containing a request or response self.forward_other_queue.append(msrpdata) self._maybe_pause_read() self.start_sending() def start_sending(self): if self.state not in ["CONNECTING", "DISCONNECTED"] and not self.registered: # self.logger.debug('Starting transmission') self.registered = True self.protocol.transport.registerProducer(self, False) def _stop_sending(self): # self.logger.debug('Empty queues, halting transmission') self.registered = False self.protocol.transport.unregisterProducer() if self.state == "DISCONNECTING": self.state = "DISCONNECTED" self.protocol.transport.loseConnection() def disconnect(self): # self.logger.debug('Disconnecting when possible') if self.invalid_timer and self.invalid_timer.active(): self.invalid_timer.cancel() self.invalid_timer = None if self.state == "NEW": self.protocol.transport.loseConnection() self.state = "DISCONNECTED" elif self.state == "UNBOUND": del self.relay.unbound_sessions[self.session.session_id] self.protocol.transport.loseConnection() self.state = "DISCONNECTED" elif self.state == "CONNECTING": self.connector.disconnect() self.state = "DISCONNECTED" elif self.state == "ESTABLISHED": - for forwarding_data, timer in self.send_transactions.itervalues(): + for forwarding_data, timer in self.send_transactions.values(): if timer is not None: timer.cancel() report = generate_report(ResponseDownstreamTimeout.code, forwarding_data, reason="Session got disconnected") self.enqueue(report.encode()) self.send_transactions.clear() - for _, timer in self.other_transactions.itervalues(): + for _, timer in self.other_transactions.values(): if timer is not None: timer.cancel() self.other_transactions.clear() if not self.registered: self.protocol.transport.loseConnection() self.state = "DISCONNECTED" else: self.state = "DISCONNECTING" def _cleanup(self): self.session = None self.other_peer = None self.protocol = None self.relay = None - for _, timer in self.send_transactions.itervalues(): + for _, timer in self.send_transactions.values(): if timer is not None and timer.active(): timer.cancel() self.send_transactions.clear() - for _, timer in self.other_transactions.itervalues(): + for _, timer in self.other_transactions.values(): if timer is not None and timer.active(): timer.cancel() self.other_transactions.clear() self.forward_send_queue.clear() self.forward_other_queue.clear() @property def _data_bytes(self): return sum(data.bytes_in_queue for data in self.forward_send_queue) @property def _message_count(self): return len(self.forward_other_queue) + len(self.forward_send_queue) def _maybe_pause_read(self): if self.state == "ESTABLISHED" and self.other_peer.state == "ESTABLISHED" and not self.other_peer.read_paused: if self._data_bytes > 1024 * 1024 or self._message_count > 50: self.other_peer.pause_read() def _maybe_resume_read(self): if self.state == "ESTABLISHED" and self.other_peer.state == "ESTABLISHED" and self.other_peer.read_paused: if self._data_bytes <= 1024 * 1024 and self._message_count <= 50: self.other_peer.resume_read() def pause_read(self): self.read_paused = True self.protocol.transport.stopReading() def resume_read(self): self.read_paused = False self.protocol.transport.startReading() # methods implemented for IPullProducer def resumeProducing(self): if self.forward_other_queue: if self.forwarding_send_request: # self.logger.debug('Sending other message, aborting SEND') data = self.forward_send_queue.popleft() # terminate the current chunk self._send_payload(data.msrpdata_forward.encode_end("+")) timer = reactor.callLater(30, self.other_peer._cb_transaction_timeout, data) self.other_peer.send_transactions[data.msrpdata_forward.transaction_id] = (data, timer) if self.other_peer.forwarding_data is data: self.other_peer.forwarding_data = None # clone the forward data new_data = ForwardingSendData(data.msrpdata_received) new_data.continuation = data.continuation new_data.position = data.position new_data.data_queue, data.data_queue = data.data_queue, deque() new_data.msrpdata_forward = data.msrpdata_forward.clone() # adjust the byterange, create a new transaction new_data.msrpdata_forward.transaction_id = generate_transaction_id() byterange = new_data.msrpdata_forward.headers["Byte-Range"].decoded byterange[0] = new_data.position new_data.msrpdata_forward.headers["Byte-Range"].decoded = byterange self.forward_send_queue.appendleft(new_data) if self.other_peer.forwarding_data is None and new_data.continuation is None: self.other_peer.forwarding_data = new_data self.forwarding_send_request = False else: # self.logger.debug('Sending message') self._send_payload(self.forward_other_queue.popleft()) self._maybe_resume_read() elif self.forwarding_send_request: data = self.forward_send_queue[0] if data.continuation is not None and len(data.data_queue) <= 1: # self.logger.debug('Sending end of SEND message') self.forward_send_queue.popleft() chunk = data.consume_data() if chunk is not None: self._send_payload(chunk) self._send_payload(data.msrpdata_forward.encode_end(data.continuation)) if data.msrpdata_received.failure_report != 'no': timer = reactor.callLater(30, self.other_peer._cb_transaction_timeout, data) self.other_peer.send_transactions[data.msrpdata_forward.transaction_id] = (data, timer) self.forwarding_send_request = False self._maybe_resume_read() else: chunk = data.consume_data() if chunk is None: self._stop_sending() else: # self.logger.debug('Sending data for SEND message') self._send_payload(chunk) data.position += len(chunk) self._maybe_resume_read() elif self.forward_send_queue: # self.logger.debug('Sending headers for SEND message') data = self.forward_send_queue[0] self.forwarding_send_request = True self._send_payload(data.msrpdata_forward.encode_start()) else: self._stop_sending() def stopProducing(self): pass def _send_payload(self, data): + data = data.encode() if self.session is not None: if self is self.session.source: self.session.upstream_bytes += len(data) else: self.session.downstream_bytes += len(data) self.protocol.transport.write(data) class Session(object): def __init__(self, source, session_id, expire, username, realm): self.source = source self.destination = None self.session_id = session_id self.expire = expire self.username = username self.realm = realm self.downstream_bytes = 0 self.upstream_bytes = 0 diff --git a/msrp/tls.py b/msrp/tls.py index 42bbcaf..7584f6a 100644 --- a/msrp/tls.py +++ b/msrp/tls.py @@ -1,48 +1,48 @@ __all__ = ['Certificate', 'PrivateKey'] from gnutls.crypto import X509Certificate, X509PrivateKey from application.process import process class _FileError(Exception): pass def file_content(file): path = process.configuration.file(file) if path is None: raise _FileError("File '%s' does not exist" % file) try: f = open(path, 'rt') except: raise _FileError("File '%s' could not be open" % file) try: return f.read() finally: f.close() class Certificate(object): """Configuration data type. Used to create a gnutls.crypto.X509Certificate object from a file given in the configuration file.""" def __new__(cls, value): if isinstance(value, str): try: return X509Certificate(file_content(value)) - except Exception, e: + except Exception as e: raise ValueError("Certificate file '%s' could not be loaded: %s" % (value, str(e))) else: raise TypeError('value should be a string') class PrivateKey(object): """Configuration data type. Used to create a gnutls.crypto.X509PrivateKey object from a file given in the configuration file.""" def __new__(cls, value): if isinstance(value, str): try: return X509PrivateKey(file_content(value)) - except Exception, e: + except Exception as e: raise ValueError("Private key file '%s' could not be loaded: %s" % (value, str(e))) else: raise TypeError('value should be a string') diff --git a/msrprelay b/msrprelay index c24b41d..cea5d11 100755 --- a/msrprelay +++ b/msrprelay @@ -1,89 +1,89 @@ -#!/usr/bin/python2 +#!/usr/bin/env python3 """MSRP Relay""" if __name__ == '__main__': import msrp import sys import signal from application import log from application.process import process, ProcessError from argparse import ArgumentParser name = 'msrprelay' fullname = 'MSRP Relay' description = 'An open source MSRP Relay' process.configuration.user_directory = None process.configuration.subdirectory = name # process.runtime.subdirectory = name parser = ArgumentParser(usage='%(prog)s [options]') parser.add_argument('--version', action='version', version='%(prog)s {}'.format(msrp.__version__)) parser.add_argument('--systemd', action='store_true', help='run as a systemd simple service and log to journal') parser.add_argument('--no-fork', action='store_false', dest='fork', help='run in the foreground and log to the terminal') parser.add_argument('--config-dir', dest='config_directory', default=None, help='the configuration directory ({})'.format(process.configuration.system_directory), metavar='PATH') parser.add_argument('--runtime-dir', dest='runtime_directory', default=None, help='the runtime directory ({})'.format(process.runtime.directory), metavar='PATH') parser.add_argument('--debug', action='store_true', help='enable verbose logging') parser.add_argument('--debug-memory', action='store_true', help='enable memory debugging') options = parser.parse_args() log.Formatter.prefix_format = '{record.levelname:<8s} ' if options.config_directory is not None: process.configuration.local_directory = options.config_directory if options.runtime_directory is not None: process.runtime.directory = options.runtime_directory if options.systemd: from systemd.journal import JournalHandler log.set_handler(JournalHandler(SYSLOG_IDENTIFIER=name)) log.capture_output() elif options.fork: try: process.daemonize(pidfile='{}.pid'.format(name)) - except ProcessError, e: + except ProcessError as e: log.critical('Cannot start %s: %s' % (fullname, e)) sys.exit(1) log.use_syslog(name) log.info('Starting %s %s' % (fullname, msrp.__version__)) try: process.wait_for_network(wait_time=10, wait_message='Waiting for network to become available...') except KeyboardInterrupt: sys.exit(0) except RuntimeError as e: log.critical('Cannot start %s: %s' % (fullname, e)) sys.exit(1) from msrp.relay import Relay, RelayConfig log.level.current = log.level.DEBUG if options.debug else RelayConfig.log_level if options.debug_memory: from application.debug.memory import memory_dump try: relay = Relay() - except Exception, e: + except Exception as e: log.critical('Failed to create %s: %s' % (fullname, e)) if e.__class__ is not RuntimeError: log.exception() sys.exit(1) process.signals.add_handler(signal.SIGHUP, lambda signum, frame: relay.reload()) try: relay.run() - except Exception, e: + except Exception as e: log.critical('Failed to run %s: %s' % (fullname, e)) if e.__class__ is not RuntimeError: log.exception() sys.exit(1) if options.debug_memory: memory_dump() diff --git a/setup.py b/setup.py index 5c415d4..c8f7761 100755 --- a/setup.py +++ b/setup.py @@ -1,36 +1,36 @@ -#!/usr/bin/python2 +#!/usr/bin/env python3 import msrp import os from distutils.core import setup def find_packages(toplevel): return [directory.replace(os.path.sep, '.') for directory, sub_dirs, files in os.walk(toplevel) if '__init__.py' in files] setup( name='msrprelay', version=msrp.__version__, - description='A MSRP Relay.', + description='Python implementation of MSRP Relay (RFC4976)', url='http://msrprelay.org/', author='AG Projects', author_email='support@ag-projects.com', license='GPLv2', platforms=['Platform Independent'], classifiers=[ - 'Development Status :: 4 - Beta', + 'Development Status :: 5 - Production', 'Intended Audience :: Service Providers', 'License :: OSI Approved :: GNU General Public License (GPL)', 'Operating System :: OS Independent', 'Programming Language :: Python', ], data_files=[('/etc/msrprelay', ['config.ini.sample'])], packages=find_packages('msrp'), scripts=['msrprelay'] ) diff --git a/test/msrp_receive_file.py b/test/msrp_receive_file.py index 3dcc2c2..6f7a6a0 100755 --- a/test/msrp_receive_file.py +++ b/test/msrp_receive_file.py @@ -1,151 +1,151 @@ #!/usr/bin/python2 import sys sys.path.append(".") sys.path.append("..") import time from base64 import b64encode from getpass import getpass from twisted.names.srvconnect import SRVConnector from twisted.internet.protocol import ClientFactory from twisted.internet import reactor from gnutls.interfaces.twisted import TLSContext, X509Credentials from msrp.protocol import * from msrp.digest import process_www_authenticate from msrp.responses import * rand_source = open("/dev/urandom") def generate_transaction_id(): return b64encode(rand_source.read(12), "+-") class MSRPFileReceiverFactory(ClientFactory): protocol = MSRPProtocol def __init__(self, username, password, relay_uri): self.uri = URI("localhost", use_tls = True, port = 12345, session_id = b64encode(rand_source.read(12))) self.username = username self.password = password self.relay_uri = relay_uri self.byte_count = 0 self.bytes_expected = 0 self.do_on_start = None self.do_on_data = None self.do_on_end = None def get_peer(self, protocol): self.protocol = protocol reactor.callLater(0, self._send_auth1) return self def data_start(self, msrpdata): if self.do_on_start: self.do_on_start(msrpdata) def write_chunk(self, data): self.byte_count += len(data) - print "received %d of %d bytes" % (self.byte_count, self.bytes_expected) + print("received %d of %d bytes" % (self.byte_count, self.bytes_expected)) if self.do_on_data: self.do_on_data(data) def data_end(self, continuation): if self.do_on_end: self.do_on_end(continuation) def connection_lost(self, reason): - print "Connection lost!" + print("Connection lost!") def _send_auth1(self): - print "Sending initial AUTH" + print("Sending initial AUTH") msrpdata = MSRPData(generate_transaction_id(), method = "AUTH") msrpdata.add_header(ToPathHeader([self.relay_uri])) msrpdata.add_header(FromPathHeader([self.uri])) self.protocol.transport.write(msrpdata.encode()) self.do_on_start = self._send_auth2 def _send_auth2(self, msrpdata): - print "Got challenge, sending response AUTH" + print("Got challenge, sending response AUTH") auth, rsp_auth = process_www_authenticate(self.username, self.password, "AUTH", str(self.relay_uri), **msrpdata.headers["WWW-Authenticate"].decoded) msrpdata = MSRPData(generate_transaction_id(), method = "AUTH") msrpdata.add_header(ToPathHeader([self.relay_uri])) msrpdata.add_header(FromPathHeader([self.uri])) msrpdata.add_header(AuthorizationHeader(auth)) self.protocol.transport.write(msrpdata.encode()) self.do_on_start = self._get_path def _get_path(self, msrpdata): if msrpdata.code != 200: - print "Failed to authenticate!" + print("Failed to authenticate!") if msrpdata.comment: - print msrpdata.comment + print(msrpdata.comment) self.protocol.transport.loseConnection() return sdp_path = list(reversed(msrpdata.headers["Use-Path"].decoded)) + [self.uri] - print "Path to send in SDP:\n%s" % " ".join(str(uri) for uri in sdp_path) - self.full_to_path = " ".join(str(uri) for uri in msrpdata.headers["Use-Path"].decoded) + " " + raw_input("Destination path: ") + print("Path to send in SDP:\n%s" % " ".join(str(uri) for uri in sdp_path)) + self.full_to_path = " ".join(str(uri) for uri in msrpdata.headers["Use-Path"].decoded) + " " + input("Destination path: ") self.do_on_start = self._start_time def _start_time(self, msrpdata): filename = msrpdata.headers["Content-Disposition"].decoded[1]["filename"] - print 'Receiving file "%s"' % filename + print('Receiving file "%s"' % filename) self.outfile= open(msrpdata.headers["Content-Disposition"].decoded[1]["filename"], "wb") self.start_time = time.time() total = msrpdata.headers["Byte-Range"].decoded[2] if total: self.bytes_expected = total self.do_on_start = None self.do_on_data = self._receive_data self.do_on_end = self._quit def _receive_data(self, data): self.outfile.write(data) def _quit(self, continuation): if continuation == "$": duration = time.time() - self.start_time speed = self.byte_count / duration / 1024 if self.byte_count == self.bytes_expected: - print "File transfer completed successfully." + print("File transfer completed successfully.") else: - print "File transfer aborted prematurely!" - print "Received %d bytes in %.0f seconds, (%.2f kb/s)" % (self.byte_count, duration, speed) + print("File transfer aborted prematurely!") + print("Received %d bytes in %.0f seconds, (%.2f kb/s)" % (self.byte_count, duration, speed)) self.protocol.transport.loseConnection() def clientConnectionFailed(self, connector, err): - print "Connection failed" - print err.value + print("Connection failed") + print(err.value) reactor.callLater(0, reactor.stop) def clientConnectionLost(self, connector, err): - print "Connection lost" - print err.value + print("Connection lost") + print(err.value) reactor.callLater(0, reactor.stop) if __name__ == "__main__": if len(sys.argv) < 2 or len(sys.argv) > 4: - print "Usage: %s user@domain [relay-hostname [relay-port]]" % sys.argv[0] - print "If the hostname and port are not specified, the MSRP relay will be discovered" - print "through the the _msrps._tcp.domain SRV record. If a hostname is specified but" - print "no port, the default port of 2855 will be used." + print("Usage: %s user@domain [relay-hostname [relay-port]]" % sys.argv[0]) + print("If the hostname and port are not specified, the MSRP relay will be discovered") + print("through the the _msrps._tcp.domain SRV record. If a hostname is specified but") + print("no port, the default port of 2855 will be used.") else: username, domain = sys.argv[1].split("@", 1) cred = X509Credentials(None, None) cred.verify_peer = False ctx = TLSContext(cred) password = getpass() if len(sys.argv) == 2: factory = MSRPFileReceiverFactory(username, password, URI(domain, use_tls=True)) connector = SRVConnector(reactor, "msrps", domain, factory, connectFuncName="connectTLS", connectFuncArgs=[ctx]) connector.connect() else: relay_host = sys.argv[2] if len(sys.argv) == 4: relay_port = int(sys.argv[3]) else: relay_port = 2855 factory = MSRPFileReceiverFactory(username, password, URI(relay_host, port=relay_port, use_tls=True)) reactor.connectTLS(relay_host, relay_port, factory, ctx) reactor.run() diff --git a/test/msrp_send_file.py b/test/msrp_send_file.py index f76151c..e709e7e 100755 --- a/test/msrp_send_file.py +++ b/test/msrp_send_file.py @@ -1,161 +1,161 @@ #!/usr/bin/python2 import sys sys.path.append(".") sys.path.append("..") import time import os from base64 import b64encode from getpass import getpass from twisted.names.srvconnect import SRVConnector from twisted.internet.protocol import ClientFactory from twisted.internet import reactor from gnutls.interfaces.twisted import TLSContext, X509Credentials from msrp.protocol import * from msrp.digest import process_www_authenticate from msrp.responses import * rand_source = open("/dev/urandom") BLOCK_SIZE = 64 * 1024 def generate_transaction_id(): return b64encode(rand_source.read(12), "+-") class MSRPFileSenderFactory(ClientFactory): protocol = MSRPProtocol def __init__(self, username, password, relay_uri, infile, filename): self.uri = URI("localhost", use_tls = True, port = 12345, session_id = b64encode(rand_source.read(12))) self.username = username self.password = password self.relay_uri = relay_uri self.infile = infile self.filename = filename self.byte_count = 0 self.do_on_start = None self.do_on_data = None self.do_on_end = None self.start_time = None self.complete = False def get_peer(self, protocol): self.protocol = protocol reactor.callLater(0, self._send_auth1) return self def data_start(self, msrpdata): if self.do_on_start: self.do_on_start(msrpdata) def write_chunk(self, data): self.byte_count += len(data) - print "received %d bytes, total %d" % (len(data), self.byte_count) + print("received %d bytes, total %d" % (len(data), self.byte_count)) if self.do_on_data: self.do_on_data(data) def data_end(self, continuation): if self.do_on_end: self.do_on_end(continuation) def connection_lost(self, reason): - print "Connection lost!" + print("Connection lost!") if self.complete: duration = time.time() - self.start_time speed = self.file_size / duration / 1024 - print "Sent %d bytes in %.0f seconds, (%.2f kb/s)" % (self.file_size, duration, speed) + print("Sent %d bytes in %.0f seconds, (%.2f kb/s)" % (self.file_size, duration, speed)) else: - print "File transfer was aborted prematurely." + print("File transfer was aborted prematurely.") def _send_auth1(self): - print "Sending initial AUTH" + print("Sending initial AUTH") msrpdata = MSRPData(generate_transaction_id(), method = "AUTH") msrpdata.add_header(ToPathHeader([self.relay_uri])) msrpdata.add_header(FromPathHeader([self.uri])) self.protocol.transport.write(msrpdata.encode()) self.do_on_start = self._send_auth2 def _send_auth2(self, msrpdata): - print "Got challenge, sending response AUTH" + print("Got challenge, sending response AUTH") auth, rsp_auth = process_www_authenticate(self.username, self.password, "AUTH", str(self.relay_uri), **msrpdata.headers["WWW-Authenticate"].decoded) msrpdata = MSRPData(generate_transaction_id(), method = "AUTH") msrpdata.add_header(ToPathHeader([self.relay_uri])) msrpdata.add_header(FromPathHeader([self.uri])) msrpdata.add_header(AuthorizationHeader(auth)) self.protocol.transport.write(msrpdata.encode()) self.do_on_start = self._get_path def _get_path(self, msrpdata): if msrpdata.code != 200: - print "Failed to authenticate!" + print("Failed to authenticate!") if msrpdata.comment: - print msrpdata.comment + print(msrpdata.comment) self.protocol.transport.loseConnection() return sdp_path = list(reversed(msrpdata.headers["Use-Path"].decoded)) + [self.uri] - print "Path to send in SDP:\n%s" % " ".join(str(uri) for uri in sdp_path) - self.full_to_path = " ".join(str(uri) for uri in msrpdata.headers["Use-Path"].decoded) + " " + raw_input("Destination path: ") - print 'Starting transmission of "%s"' % self.filename + print("Path to send in SDP:\n%s" % " ".join(str(uri) for uri in sdp_path)) + self.full_to_path = " ".join(str(uri) for uri in msrpdata.headers["Use-Path"].decoded) + " " + input("Destination path: ") + print('Starting transmission of "%s"' % self.filename) self.do_on_start = None self.infile.seek(0, 2) self.file_size = self.infile.tell() self.infile.seek(0, 0) msrpdata = MSRPData(generate_transaction_id(), method = "SEND") msrpdata.add_header(FromPathHeader([self.uri])) msrpdata.add_header(MSRPHeader("To-Path", self.full_to_path)) msrpdata.add_header(MessageIDHeader("1")) msrpdata.add_header(ByteRangeHeader([1, self.file_size, self.file_size])) msrpdata.add_header(ContentTypeHeader("binary/octet-stream")) msrpdata.add_header(FailureReportHeader("no")) msrpdata.add_header(ContentDispositionHeader(["attachment", {"filename": self.filename}])) self.start_time = time.time() self.protocol.transport.write(msrpdata.encode_start()) sent = 0 for i in range(0, self.file_size, BLOCK_SIZE): - print "sent %d of %d bytes" % (sent, self.file_size) + print("sent %d of %d bytes" % (sent, self.file_size)) sent += i data = self.infile.read(BLOCK_SIZE) self.protocol.transport.write(data) - print "File transfer completed." + print("File transfer completed.") self.complete = True self.protocol.transport.write(msrpdata.encode_end("$")) def clientConnectionFailed(self, connector, err): - print "Connection failed" - print err.value + print("Connection failed") + print(err.value) reactor.callLater(0, reactor.stop) def clientConnectionLost(self, connector, err): - print "Connection lost" - print err.value + print("Connection lost") + print(err.value) reactor.callLater(0, reactor.stop) if __name__ == "__main__": if len(sys.argv) < 3 or len(sys.argv) > 5: - print "Usage: %s infile user@domain [relay-hostname [relay-port]]" % sys.argv[0] - print "If the hostname and port are not specified, the MSRP relay will be discovered" - print "through the the _msrps._tcp.domain SRV record. If a hostname is specified but" - print "no port, the default port of 2855 will be used." + print("Usage: %s infile user@domain [relay-hostname [relay-port]]" % sys.argv[0]) + print("If the hostname and port are not specified, the MSRP relay will be discovered") + print("through the the _msrps._tcp.domain SRV record. If a hostname is specified but") + print("no port, the default port of 2855 will be used.") else: filename = sys.argv[1] infile = open(filename, "rb") username, domain = sys.argv[2].split("@", 1) cred = X509Credentials(None, None) cred.verify_peer = False ctx = TLSContext(cred) password = getpass() if len(sys.argv) == 3: factory = MSRPFileSenderFactory(username, password, URI(domain, use_tls = True), infile, filename.split(os.path.sep)[-1]) connector = SRVConnector(reactor, "msrps", domain, factory, connectFuncName="connectTLS", connectFuncArgs=[ctx]) connector.connect() else: relay_host = sys.argv[3] if len(sys.argv) == 5: relay_port = int(sys.argv[4]) else: relay_port = 2855 factory = MSRPFileSenderFactory(username, password, URI(relay_host, port = relay_port, use_tls = True), infile, filename.split(os.path.sep)[-1]) reactor.connectTLS(relay_host, relay_port, factory, ctx) reactor.run() diff --git a/tls/README b/tls/README index 53748c5..ebbd328 100644 --- a/tls/README +++ b/tls/README @@ -1,19 +1,26 @@ These scripts provide two ways to generate a certificate: + CA method --------- + - Generate a self-signed CA certificate and key pair using: + ./gen_ca_creds.sh . -- Use the CA certificate and key pair to generate a relay certificate and key - pair: + +- Use the CA certificate and key pair to generate a relay + certificate and key pair: + ./gen_relay_creds_ca.sh The CA certificate could then be handed to the client and used for certificate verification. + Self-signed method ------------------ + - Generate a self-signed certificate and key pair: ./gen_relay_creds_self.sh This should be sufficient for most relays.