New upstream version 0.16.0

CONTRIBUTING

@@ -1,52 +0,0 @@
-All functionality should be available in pure Python. Optional C
-implementations may be written for performance reasons, but should never
-replace the Python implementation. The C implementations should follow the
-kernel/git coding style.
-
-Where possible include updates to NEWS along with your improvements.
-
-New functionality and bug fixes should be accompanied by matching unit tests.
-
-Coding style
-------------
-Where possible, please follow PEP8 with regard to coding style.
-
-Furthermore, triple-quotes should always be """, single quotes are ' unless
-using " would result in less escaping within the string.
-
-Public methods, functions and classes should all have doc strings. Please use
-epydoc style docstrings to document parameters and return values.
-You can generate the documentation by running "make doc".
-
-Running the tests
------------------
-To run the testsuite, you should be able to simply run "make check". This
-will run the tests using unittest on Python 2.7 and higher, and using
-unittest2 (which you will need to have installed) on older versions of Python.
-
- $ make check
-
-Tox configuration is also present as well as a Travis configuration file.
-
-String Types
-------------
-Like Linux, Git treats filenames as arbitrary bytestrings. There is no prescribed
-encoding for these strings, and although it is fairly common to use UTF-8, any
-raw byte strings are supported.
-
-For this reason, Dulwich internally treats git-based filenames as bytestrings. It is up
-to the Dulwich API user to encode and decode them if necessary.
-
-* git-repository related filenames: bytes
-* object sha1 digests (20 bytes long): bytes
-* object sha1 hexdigests (40 bytes long): str (bytestrings on python2, strings on python3)
-
-Merge requests
---------------
-Please either send pull requests to the author (jelmer@jelmer.uk) or create new pull
-requests on GitHub.
-
-Licensing
----------
-All contributions should be made under the same license that Dulwich itself comes under:
-both Apache License, version 2.0 or later and GNU General Public License, version 2.0 or later.

NEWS

@@ -1,3 +1,38 @@
+0.16.0	2016-12-24
+
+ IMPROVEMENTS
+
+  * Add support for worktrees. See `git-worktree(1)` and
+    `gitrepository-layout(5)`. (Laurent Rineau)
+
+  * Add support for `commondir` file in Git control
+    directories. (Laurent Rineau)
+
+  * Add support for passwords in HTTP URLs.
+    (Jon Bain, Mika Mäenpää)
+
+  * Add `release_robot` script to contrib,
+    allowing easy finding of current version based on Git tags.
+    (Mark Mikofski)
+
+  * Add ``Blob.splitlines`` method.
+    (Jelmer Vernooij)
+
+ BUG FIXES
+
+  * Fix ``porcelain.reset`` to respect the comittish argument.
+    (Koen Martens)
+
+  * Fix handling of ``Commit.tree`` being set to an actual
+    tree object rather than a tree id. (Jelmer Vernooij)
+
+  * Return remote refs from LocalGitClient.fetch_pack(),
+    consistent with the documentation for that method.
+    (#461, Jelmer Vernooij)
+
+  * Fix handling of unknown URL schemes in get_transport_and_path.
+    (#465, Jelmer Vernooij)
+
 0.15.0	2016-10-09
 
  BUG FIXES

PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.1
 Name: dulwich
-Version: 0.15.0
+Version: 0.16.0
 Summary: Python Git Library
 Home-page: https://www.dulwich.io/
 Author: Jelmer Vernooij

README.swift

@@ -1,133 +0,0 @@
-Openstack Swift as backend for Dulwich
-======================================
-Fabien Boucher <fabien.boucher@enovance.com>
-
-The module dulwich/contrib/swift.py implements dulwich.repo.BaseRepo
-in order to being compatible with Openstack Swift.
-We can then use Dulwich as server (Git server) and instead of using
-a regular POSIX file system to store repository objects we use the
-object storage Swift via its own API.
-
-c Git client <---> Dulwich server <---> Openstack Swift API
-
-This implementation is still a work in progress and we can say that
-is a Beta version so you need to be prepared to find bugs.
-
-Configuration file
-------------------
-
-We need to provide some configuration values in order to let Dulwich
-talk and authenticate against Swift. The following config file must
-be used as template:
-
-    [swift]
-    # Authentication URL (Keystone or Swift)
-    auth_url = http://127.0.0.1:5000/v2.0
-    # Authentication version to use
-    auth_ver = 2
-    # The tenant and username separated by a semicolon
-    username = admin;admin
-    # The user password
-    password = pass
-    # The Object storage region to use (auth v2) (Default RegionOne)
-    region_name = RegionOne
-    # The Object storage endpoint URL to use (auth v2) (Default internalURL)
-    endpoint_type = internalURL
-    # Concurrency to use for parallel tasks (Default 10)
-    concurrency = 10
-    # Size of the HTTP pool (Default 10)
-    http_pool_length = 10
-    # Timeout delay for HTTP connections (Default 20)
-    http_timeout = 20
-    # Chunk size to read from pack (Bytes) (Default 12228)
-    chunk_length = 12228
-    # Cache size (MBytes) (Default 20)
-    cache_length = 20
-
-
-Note that for now we use the same tenant to perform the requests
-against Swift. Therefor there is only one Swift account used
-for storing repositories. Each repository will be contained in
-a Swift container.
-
-How to start unittest
----------------------
-
-There is no need to have a Swift cluster running to run the unitests.
-Just run the following command in the Dulwich source directory:
-
-    $ PYTHONPATH=. python -m dulwich.contrib.test_swift
-
-How to start functional tests
------------------------------
-
-We provide some basic tests to perform smoke tests against a real Swift
-cluster. To run those functional tests you need a properly configured
-configuration file. The tests can be run as follow:
-
-    $ DULWICH_SWIFT_CFG=/etc/swift-dul.conf PYTHONPATH=. python -m dulwich.contrib.test_swift_smoke
-
-How to install
---------------
-
-Install the Dulwich library via the setup.py. The dependencies will be
-automatically retrieved from pypi:
-
-    $ python ./setup.py install
-
-How to run the server
----------------------
-
-Start the server using the following command:
-
-    $ python -m dulwich.contrib.swift daemon -c /etc/swift-dul.conf -l 127.0.0.1
-
-Note that a lot of request will be performed against the Swift
-cluster so it is better to start the Dulwich server as close
-as possible of the Swift proxy. The best solution is to run
-the server on the Swift proxy node to reduce the latency.
-
-How to use
-----------
-
-Once you have validated that the functional tests is working as expected and
-the server is running we can init a bare repository. Run this
-command with the name of the repository to create:
-
-    $ python -m dulwich.contrib.swift init -c /etc/swift-dul.conf edeploy
-
-The repository name will be the container that will contain all the Git
-objects for the repository. Then standard c Git client can be used to
-perform operations againt this repository.
-
-As an example we can clone the previously empty bare repository:
-
-    $ git clone git://localhost/edeploy
-
-Then push an existing project in it:
-
-    $ git clone https://github.com/enovance/edeploy.git edeployclone
-    $ cd edeployclone
-    $ git remote add alt git://localhost/edeploy
-    $ git push alt master
-    $ git ls-remote alt
-    9dc50a9a9bff1e232a74e365707f22a62492183e        HEAD
-    9dc50a9a9bff1e232a74e365707f22a62492183e        refs/heads/master
-
-The other Git commands can be used the way you do usually against
-a regular repository.
-
-Note the daemon subcommands starts a Git server listening for the
-Git protocol. Therefor there is no authentication or encryption
-at all between the cGIT client and the GIT server (Dulwich).
-
-Note on the .info file for pack object
---------------------------------------
-
-The Swift interface of Dulwich relies only on the pack format
-to store Git objects. Instead of using only an index (pack-sha.idx)
-along with the pack, we add a second file (pack-sha.info). This file
-is automatically created when a client pushes some references on the
-repository. The purpose of this file is to speed up pack creation
-server side when a client fetches some references. Currently this
-.info format is not optimized and may change in future.

bin/dulwich

@@ -23,8 +23,8 @@
 
 """Simple command-line interface to Dulwich>
 
-This is a very simple command-line wrapper for Dulwich. It is by 
-no means intended to be a full-blown Git command-line interface but just 
+This is a very simple command-line wrapper for Dulwich. It is by
+no means intended to be a full-blown Git command-line interface but just
 a way to test Dulwich.
 """
 
@@ -95,12 +95,16 @@ def cmd_fetch(args):
 
 
 def cmd_log(args):
-    opts, args = getopt(args, "", [])
-    if len(args) > 0:
-        path = args.pop(0)
-    else:
-        path = "."
-    porcelain.log(repo=path, outstream=sys.stdout)
+    parser = optparse.OptionParser()
+    parser.add_option("--reverse", dest="reverse", action="store_true",
+                      help="Reverse order in which entries are printed")
+    parser.add_option("--name-status", dest="name_status", action="store_true",
+                      help="Print name/status for each changed file")
+    options, args = parser.parse_args(args)
+
+    porcelain.log(".", paths=args, reverse=options.reverse,
+                  name_status=options.name_status,
+                  outstream=sys.stdout)
 
 
 def cmd_diff(args):
@@ -397,6 +401,26 @@ def cmd_pack_objects(args):
         f.close()
 
 
+def cmd_help(args):
+    parser = optparse.OptionParser()
+    parser.add_option("-a", "--all", dest="all",
+                      action="store_true",
+                      help="List all commands.")
+    options, args = parser.parse_args(args)
+
+    if options.all:
+        print('Available commands:')
+        for cmd in sorted(commands):
+            print('  %s' % cmd)
+    else:
+        print("""\
+The dulwich command line tool is currently a very basic frontend for the
+Dulwich python module. For full functionality, please see the API reference.
+
+For a list of supported commands, see 'dulwich help -a'.
+""")
+
+
 commands = {
     "add": cmd_add,
     "archive": cmd_archive,
@@ -410,6 +434,7 @@ commands = {
     "dump-index": cmd_dump_index,
     "fetch-pack": cmd_fetch_pack,
     "fetch": cmd_fetch,
+    "help": cmd_help,
     "init": cmd_init,
     "log": cmd_log,
     "ls-remote": cmd_ls_remote,

docs/tutorial/encoding.txt

@@ -0,0 +1,26 @@
+Encoding
+========
+
+You will notice that all lower-level functions in Dulwich take byte strings
+rather than unicode strings. This is intentional.
+
+Although `C git`_ recommends the use of UTF-8 for encoding, this is not
+strictly enforced and C git treats filenames as sequences of non-NUL bytes.
+There are repositories in the wild that use non-UTF-8 encoding for filenames
+and commit messages.
+
+.. _C git: https://github.com/git/git/blob/master/Documentation/i18n.txt
+
+The library should be able to read *all* existing git repositories,
+irregardless of what encoding they use. This is the main reason why Dulwich
+does not convert paths to unicode strings.
+
+A further consideration is that converting back and forth to unicode
+is an extra performance penalty. E.g. if you are just iterating over file
+contents, there is no need to consider encoded strings. Users of the library
+may have specific assumptions they can make about the encoding - e.g. they
+could just decide that all their data is latin-1, or the default Python
+encoding.
+
+Higher level functions, such as the porcelain in dulwich.porcelain, will
+automatically convert unicode strings to UTF-8 bytestrings.

docs/tutorial/index.txt

@@ -8,6 +8,7 @@ Tutorial
    :maxdepth: 2
 
    introduction
+   encoding 
    file-format
    repo
    object-store

dulwich.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.1
 Name: dulwich
-Version: 0.15.0
+Version: 0.16.0
 Summary: Python Git Library
 Home-page: https://www.dulwich.io/
 Author: Jelmer Vernooij

dulwich.egg-info/SOURCES.txt

@@ -1,13 +1,11 @@
 .testr.conf
 .travis.yml
 AUTHORS
-CONTRIBUTING
 COPYING
 MANIFEST.in
 Makefile
 NEWS
 README.md
-README.swift
 TODO
 appveyor.yml
 dulwich.cfg
@@ -25,6 +23,7 @@ docs/performance.txt
 docs/protocol.txt
 docs/tutorial/Makefile
 docs/tutorial/conclusion.txt
+docs/tutorial/encoding.txt
 docs/tutorial/file-format.txt
 docs/tutorial/index.txt
 docs/tutorial/introduction.txt
@@ -69,6 +68,7 @@ dulwich.egg-info/dependency_links.txt
 dulwich.egg-info/top_level.txt
 dulwich/contrib/__init__.py
 dulwich/contrib/paramiko_vendor.py
+dulwich/contrib/release_robot.py
 dulwich/contrib/swift.py
 dulwich/contrib/test_swift.py
 dulwich/contrib/test_swift_smoke.py

dulwich/__init__.py

@@ -22,4 +22,4 @@
 
 """Python implementation of the Git file formats and protocols."""
 
-__version__ = (0, 15, 0)
+__version__ = (0, 16, 0)

dulwich/client.py

@@ -222,6 +222,15 @@ class GitClient(object):
         """
         raise NotImplementedError(self.get_url)
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        """Create an instance of this client from a urlparse.parsed object.
+
+        :param parsedurl: Result of urlparse.urlparse()
+        :return: A `GitClient` object
+        """
+        raise NotImplementedError(cls.from_parsedurl)
+
     def send_pack(self, path, determine_wants, generate_pack_contents,
                   progress=None, write_pack=write_pack_objects):
         """Upload a pack to a remote repository.
@@ -651,6 +660,10 @@ class TCPGitClient(TraditionalGitClient):
         self._port = port
         super(TCPGitClient, self).__init__(**kwargs)
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        return cls(parsedurl.hostname, port=parsedurl.port, **kwargs)
+
     def get_url(self, path):
         netloc = self._host
         if self._port is not None and self._port != TCP_GIT_PORT:
@@ -751,6 +764,10 @@ class SubprocessGitClient(TraditionalGitClient):
             del kwargs['stderr']
         super(SubprocessGitClient, self).__init__(**kwargs)
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        return cls(**kwargs)
+
     git_command = None
 
     def _connect(self, service, path):
@@ -785,6 +802,10 @@ class LocalGitClient(GitClient):
     def get_url(self, path):
         return urlparse.urlunsplit(('file', '', path, '', ''))
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        return cls(**kwargs)
+
     def send_pack(self, path, determine_wants, generate_pack_contents,
                   progress=None, write_pack=write_pack_objects):
         """Upload a pack to a remote repository.
@@ -866,6 +887,7 @@ class LocalGitClient(GitClient):
             if objects_iter is None:
                 return
             write_pack_objects(ProtocolFile(None, pack_data), objects_iter)
+            return r.get_refs()
 
     def get_refs(self, path):
         """Retrieve the current refs from a git smart server."""
@@ -959,6 +981,11 @@ class SSHGitClient(TraditionalGitClient):
 
         return urlparse.urlunsplit(('ssh', netloc, path, '', ''))
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        return cls(host=parsedurl.hostname, port=parsedurl.port,
+                   username=parsedurl.username, **kwargs)
+
     def _get_cmd_path(self, cmd):
         cmd = self.alternative_paths.get(cmd, b'git-' + cmd)
         assert isinstance(cmd, bytes)
@@ -1004,18 +1031,35 @@ def default_urllib2_opener(config):
 
 class HttpGitClient(GitClient):
 
-    def __init__(self, base_url, dumb=None, opener=None, config=None, **kwargs):
+    def __init__(self, base_url, dumb=None, opener=None, config=None,
+                 username=None, password=None, **kwargs):
         self._base_url = base_url.rstrip("/") + "/"
+        self._username = username
+        self._password = password
         self.dumb = dumb
         if opener is None:
             self.opener = default_urllib2_opener(config)
         else:
             self.opener = opener
+        if username is not None:
+            pass_man = urllib2.HTTPPasswordMgrWithDefaultRealm()
+            pass_man.add_password(None, base_url, username, password)
+            self.opener.add_handler(urllib2.HTTPBasicAuthHandler(pass_man))
         GitClient.__init__(self, **kwargs)
 
     def get_url(self, path):
         return self._get_url(path).rstrip("/")
 
+    @classmethod
+    def from_parsedurl(cls, parsedurl, **kwargs):
+        auth, host = urllib2.splituser(parsedurl.netloc)
+        password = parsedurl.password
+        username = parsedurl.username
+        # TODO(jelmer): This also strips the username
+        parsedurl = parsedurl._replace(netloc=host)
+        return cls(urlparse.urlunparse(parsedurl),
+                   password=password, username=username, **kwargs)
+
     def __repr__(self):
         return "%s(%r, dumb=%r)" % (type(self).__name__, self._base_url, self.dumb)
 
@@ -1189,19 +1233,19 @@ def get_transport_and_path_from_url(url, config=None, **kwargs):
     """
     parsed = urlparse.urlparse(url)
     if parsed.scheme == 'git':
-        return (TCPGitClient(parsed.hostname, port=parsed.port, **kwargs),
+        return (TCPGitClient.from_parsedurl(parsed, **kwargs),
                 parsed.path)
     elif parsed.scheme in ('git+ssh', 'ssh'):
         path = parsed.path
         if path.startswith('/'):
             path = parsed.path[1:]
-        return SSHGitClient(parsed.hostname, port=parsed.port,
-                            username=parsed.username, **kwargs), path
+        return SSHGitClient.from_parsedurl(parsed, **kwargs), path
     elif parsed.scheme in ('http', 'https'):
-        return HttpGitClient(urlparse.urlunparse(parsed), config=config,
-                **kwargs), parsed.path
+        return HttpGitClient.from_parsedurl(
+            parsed, config=config, **kwargs), parsed.path
     elif parsed.scheme == 'file':
-        return default_local_git_client_cls(**kwargs), parsed.path
+        return default_local_git_client_cls.from_parsedurl(
+            parsed, **kwargs), parsed.path
 
     raise ValueError("unknown scheme '%s'" % parsed.scheme)
 
@@ -1229,12 +1273,16 @@ def get_transport_and_path(location, **kwargs):
 
     if ':' in location and not '@' in location:
         # SSH with no user@, zero or one leading slash.
-        (hostname, path) = location.split(':')
+        (hostname, path) = location.split(':', 1)
         return SSHGitClient(hostname, **kwargs), path
-    elif '@' in location and ':' in location:
+    elif ':' in location:
         # SSH with user@host:foo.
-        user_host, path = location.split(':')
-        user, host = user_host.rsplit('@')
+        user_host, path = location.split(':', 1)
+        if '@' in user_host:
+            user, host = user_host.rsplit('@', 1)
+        else:
+            user = None
+            host = user_host
         return SSHGitClient(host, username=user, **kwargs), path
 
     # Otherwise, assume it's a local path.

dulwich/contrib/release_robot.py

@@ -0,0 +1,110 @@
+"""Determine last version string from tags.
+
+Alternate to `Versioneer <https://pypi.python.org/pypi/versioneer/>`_ using
+`Dulwich <https://pypi.python.org/pypi/dulwich>`_ to sort tags by time from
+newest to oldest.
+
+Import this module into the package ``__init__.py`` and then set ``__version__``
+as follows::
+
+    from dulwich.contrib.release_robot import get_current_version
+
+    __version__ = get_current_version()
+    # other dunder classes like __author__, etc.
+
+This example assumes the tags have a leading "v" like "v0.3", and that the
+``.git`` folder is in the project folder that containts the package folder.
+"""
+
+from dulwich.repo import Repo
+import time
+import datetime
+import os
+import re
+import sys
+
+# CONSTANTS
+DIRNAME = os.path.abspath(os.path.dirname(__file__))
+PROJDIR = os.path.dirname(DIRNAME)
+PATTERN = '[ a-zA-Z_\-]*([\d\.]+[\-\w\.]*)'
+
+
+def get_recent_tags(projdir=PROJDIR):
+    """Get list of tags in order from newest to oldest and their datetimes.
+
+    :param projdir: path to ``.git``
+    :returns: list of (tag, [datetime, commit, author]) sorted from new to old
+    """
+    project = Repo(projdir)  # dulwich repository object
+    refs = project.get_refs()  # dictionary of refs and their SHA-1 values
+    tags = {}  # empty dictionary to hold tags, commits and datetimes
+    # iterate over refs in repository
+    for key, value in refs.iteritems():
+        obj = project.get_object(value)  # dulwich object from SHA-1
+        # check if object is tag
+        if obj.type_name != 'tag':
+            # skip ref if not a tag
+            continue
+        # strip the leading text from "refs/tag/<tag name>" to get "tag name"
+        _, tag = key.rsplit('/', 1)
+        # check if tag object is commit, altho it should always be true
+        if obj.object[0].type_name == 'commit':
+            commit = project.get_object(obj.object[1])  # commit object
+            # get tag commit datetime, but dulwich returns seconds since
+            # beginning of epoch, so use Python time module to convert it to
+            # timetuple then convert to datetime
+            tags[tag] = [
+                datetime.datetime(*time.gmtime(commit.commit_time)[:6]),
+                commit.id,
+                commit.author
+            ]
+
+    # return list of tags sorted by their datetimes from newest to oldest
+    return sorted(tags.iteritems(), key=lambda tag: tag[1][0], reverse=True)
+
+
+def get_current_version(pattern=PATTERN, projdir=PROJDIR, logger=None):
+    """Return the most recent tag, using an options regular expression pattern.
+
+    The default pattern will strip any characters preceding the first semantic
+    version. *EG*: "Release-0.2.1-rc.1" will be come "0.2.1-rc.1". If no match
+    is found, then the most recent tag is return without modification.
+
+    :param pattern: regular expression pattern with group that matches version
+    :param projdir: path to ``.git``
+    :param logger: a Python logging instance to capture exception
+    :returns: tag matching first group in regular expression pattern
+    """
+    tags = get_recent_tags(projdir)
+    try:
+        tag = tags[0][0]
+    except IndexError:
+        return
+    m = re.match(pattern, tag)
+    try:
+        current_version = m.group(1)
+    except (IndexError, AttributeError) as err:
+        if logger:
+            logger.exception(err)
+        return tag
+    return current_version
+
+
+def test_tag_pattern():
+    test_cases = {
+        '0.3': '0.3', 'v0.3': '0.3', 'release0.3': '0.3', 'Release-0.3': '0.3',
+        'v0.3rc1': '0.3rc1', 'v0.3-rc1': '0.3-rc1', 'v0.3-rc.1': '0.3-rc.1',
+        'version 0.3': '0.3', 'version_0.3_rc_1': '0.3_rc_1', 'v1': '1',
+        '0.3rc1': '0.3rc1'
+    }
+    for tc, version in test_cases.iteritems():
+        m = re.match(PATTERN, tc)
+        assert m.group(1) == version
+
+
+if __name__ == '__main__':
+    if len(sys.argv) > 1:
+        projdir = sys.argv[1]
+    else:
+        projdir = PROJDIR
+    print get_current_version(projdir=projdir)

dulwich/object_store.py

@@ -131,7 +131,7 @@ class BaseObjectStore(object):
     def add_objects(self, objects):
         """Add a set of objects to this object store.
 
-        :param objects: Iterable over a list of objects.
+        :param objects: Iterable over a list of (object, path) tuples
         """
         raise NotImplementedError(self.add_objects)
 
@@ -382,7 +382,8 @@ class PackBasedObjectStore(BaseObjectStore):
     def add_objects(self, objects):
         """Add a set of objects to this object store.
 
-        :param objects: Iterable over objects, should support __len__.
+        :param objects: Iterable over (object, path) tuples, should support
+            __len__.
         :return: Pack object of the objects written.
         """
         if len(objects) == 0:
@@ -740,7 +741,7 @@ class MemoryObjectStore(BaseObjectStore):
     def add_objects(self, objects):
         """Add a set of objects to this object store.
 
-        :param objects: Iterable over a list of objects.
+        :param objects: Iterable over a list of (object, path) tuples
         """
         for obj, path in objects:
             self.add_object(obj)

dulwich/objects.py

@@ -568,6 +568,33 @@ class Blob(ShaFile):
         """
         super(Blob, self).check()
 
+    def splitlines(self):
+        """Return list of lines in this blob.
+
+        This preserves the original line endings.
+        """
+        chunks = self.chunked
+        if not chunks:
+            return []
+        if len(chunks) == 1:
+            return chunks[0].splitlines(True)
+        remaining = None
+        ret = []
+        for chunk in chunks:
+            lines = chunk.splitlines(True)
+            if len(lines) > 1:
+                ret.append((remaining or b"") + lines[0])
+                ret.extend(lines[1:-1])
+                remaining = lines[-1]
+            elif len(lines) == 1:
+                if remaining is None:
+                    remaining = lines.pop()
+                else:
+                    remaining += lines.pop()
+        if remaining is not None:
+            ret.append(remaining)
+        return ret
+
 
 def _parse_message(chunks):
     """Parse a message with a list of fields and a body.
@@ -1154,7 +1181,7 @@ class Commit(ShaFile):
 
     def _serialize(self):
         chunks = []
-        tree_bytes = self._tree.as_raw_string() if isinstance(self._tree, Tree) else self._tree
+        tree_bytes = self._tree.id if isinstance(self._tree, Tree) else self._tree
         chunks.append(git_line(_TREE_HEADER, tree_bytes))
         for p in self._parents:
             chunks.append(git_line(_PARENT_HEADER, p))

dulwich/patch.py

@@ -29,6 +29,7 @@ import email.parser
 import time
 
 from dulwich.objects import (
+    Blob,
     Commit,
     S_ISGITLINK,
     )
@@ -152,22 +153,23 @@ def write_object_diff(f, store, old_file, new_file, diff_binary=False):
     new_path = patch_filename(new_path, b"b")
     def content(mode, hexsha):
         if hexsha is None:
-            return b''
+            return Blob.from_string(b'')
         elif S_ISGITLINK(mode):
-            return b"Submodule commit " + hexsha + b"\n"
+            return Blob.from_string(b"Submodule commit " + hexsha + b"\n")
         else:
-            return store[hexsha].data
+            return store[hexsha]
 
     def lines(content):
         if not content:
             return []
         else:
-            return content.splitlines(True)
+            return content.splitlines()
     f.writelines(gen_diff_header(
         (old_path, new_path), (old_mode, new_mode), (old_id, new_id)))
     old_content = content(old_mode, old_id)
     new_content = content(new_mode, new_id)
-    if not diff_binary and (is_binary(old_content) or is_binary(new_content)):
+    if not diff_binary and (
+            is_binary(old_content.data) or is_binary(new_content.data)):
         f.write(b"Binary files " + old_path + b" and " + new_path + b" differ\n")
     else:
         f.writelines(unified_diff(lines(old_content), lines(new_content),
@@ -215,7 +217,7 @@ def write_blob_diff(f, old_file, new_file):
     new_path = patch_filename(new_path, b"b")
     def lines(blob):
         if blob is not None:
-            return blob.data.splitlines(True)
+            return blob.splitlines()
         else:
             return []
     f.writelines(gen_diff_header(

dulwich/porcelain.py

@@ -68,6 +68,14 @@ from dulwich.archive import (
 from dulwich.client import (
     get_transport_and_path,
     )
+from dulwich.diff_tree import (
+    CHANGE_ADD,
+    CHANGE_DELETE,
+    CHANGE_MODIFY,
+    CHANGE_RENAME,
+    CHANGE_COPY,
+    RENAME_CHANGE_TYPES,
+    )
 from dulwich.errors import (
     SendPackError,
     UpdateRefsError,
@@ -76,6 +84,7 @@ from dulwich.index import get_unstaged_changes
 from dulwich.objects import (
     Commit,
     Tag,
+    format_timezone,
     parse_timezone,
     pretty_format_tree_entry,
     )
@@ -110,11 +119,13 @@ default_bytes_out_stream = getattr(sys.stdout, 'buffer', sys.stdout)
 default_bytes_err_stream = getattr(sys.stderr, 'buffer', sys.stderr)
 
 
-def encode_path(path):
+DEFAULT_ENCODING = 'utf-8'
+
+
+def encode_path(path, default_encoding=DEFAULT_ENCODING):
     """Encode a path as bytestring."""
-    # TODO(jelmer): Use something other than ascii?
     if not isinstance(path, bytes):
-        path = path.encode('ascii')
+        path = path.encode(default_encoding)
     return path
 
 
@@ -319,7 +330,7 @@ def rm(repo=".", paths=None):
         index.write()
 
 
-def commit_decode(commit, contents, default_encoding='utf-8'):
+def commit_decode(commit, contents, default_encoding=DEFAULT_ENCODING):
     if commit.encoding is not None:
         return contents.decode(commit.encoding, "replace")
     return contents.decode(default_encoding, "replace")
@@ -336,8 +347,14 @@ def print_commit(commit, decode, outstream=sys.stdout):
     if len(commit.parents) > 1:
         outstream.write("merge: " +
             "...".join([c.decode('ascii') for c in commit.parents[1:]]) + "\n")
-    outstream.write("author: " + decode(commit.author) + "\n")
-    outstream.write("committer: " + decode(commit.committer) + "\n")
+    outstream.write("Author: " + decode(commit.author) + "\n")
+    if commit.author != commit.committer:
+        outstream.write("Committer: " + decode(commit.committer) + "\n")
+
+    time_tuple = time.gmtime(commit.author_time + commit.author_timezone)
+    time_str = time.strftime("%a %b %d %Y %H:%M:%S", time_tuple)
+    timezone_str = format_timezone(commit.author_timezone).decode('ascii')
+    outstream.write("Date:   " + time_str + " " + timezone_str + "\n")
     outstream.write("\n")
     outstream.write(decode(commit.message) + "\n")
     outstream.write("\n")
@@ -414,22 +431,61 @@ def show_object(repo, obj, decode, outstream):
             }[obj.type_name](repo, obj, decode, outstream)
 
 
-def log(repo=".", outstream=sys.stdout, max_entries=None):
+def print_name_status(changes):
+    """Print a simple status summary, listing changed files.
+    """
+    for change in changes:
+        if not change:
+            continue
+        if type(change) is list:
+            change = change[0]
+        if change.type == CHANGE_ADD:
+            path1 = change.new.path
+            path2 = ''
+            kind = 'A'
+        elif change.type == CHANGE_DELETE:
+            path1 = change.old.path
+            path2 = ''
+            kind = 'D'
+        elif change.type == CHANGE_MODIFY:
+            path1 = change.new.path
+            path2 = ''
+            kind = 'M'
+        elif change.type in RENAME_CHANGE_TYPES:
+            path1 = change.old.path
+            path2 = change.new.path
+            if change.type == CHANGE_RENAME:
+                kind = 'R'
+            elif change.type == CHANGE_COPY:
+                kind = 'C'
+        yield '%-8s%-20s%-20s' % (kind, path1, path2)
+
+
+def log(repo=".", paths=None, outstream=sys.stdout, max_entries=None,
+        reverse=False, name_status=False):
     """Write commit logs.
 
     :param repo: Path to repository
+    :param paths: Optional set of specific paths to print entries for
     :param outstream: Stream to write log output to
+    :param reverse: Reverse order in which entries are printed
+    :param name_status: Print name status
     :param max_entries: Optional maximum number of entries to display
     """
     with open_repo_closing(repo) as r:
-        walker = r.get_walker(max_entries=max_entries)
+        walker = r.get_walker(
+            max_entries=max_entries, paths=paths, reverse=reverse)
         for entry in walker:
             decode = lambda x: commit_decode(entry.commit, x)
             print_commit(entry.commit, decode, outstream)
+            if name_status:
+                outstream.writelines(
+                    [l+'\n' for l in print_name_status(entry.changes())])
 
 
 # TODO(jelmer): better default for encoding?
-def show(repo=".", objects=None, outstream=sys.stdout, default_encoding='utf-8'):
+def show(repo=".", objects=None, outstream=sys.stdout,
+         default_encoding=DEFAULT_ENCODING):
     """Print the changes in a commit.
 
     :param repo: Path to repository
@@ -572,7 +628,7 @@ def reset(repo, mode, committish="HEAD"):
 
     with open_repo_closing(repo) as r:
         tree = r[committish].tree
-        r.reset_index()
+        r.reset_index(tree)
 
 
 def push(repo, remote_location, refspecs=None,
@@ -605,7 +661,7 @@ def push(repo, remote_location, refspecs=None,
                     new_refs[rh] = r.refs[lh]
             return new_refs
 
-        err_encoding = getattr(errstream, 'encoding', None) or 'utf-8'
+        err_encoding = getattr(errstream, 'encoding', None) or DEFAULT_ENCODING
         remote_location_bytes = client.get_url(path).encode(err_encoding)
         try:
             client.send_pack(path, update_refs,

dulwich/refs.py

@@ -403,8 +403,9 @@ class InfoRefsContainer(RefsContainer):
 class DiskRefsContainer(RefsContainer):
     """Refs container that reads refs from disk."""
 
-    def __init__(self, path):
+    def __init__(self, path, worktree_path=None):
         self.path = path
+        self.worktree_path = worktree_path or path
         self._packed_refs = None
         self._peeled_refs = None
 
@@ -450,7 +451,12 @@ class DiskRefsContainer(RefsContainer):
             name = name.decode(sys.getfilesystemencoding())
         if os.path.sep != "/":
             name = name.replace("/", os.path.sep)
-        return os.path.join(self.path, name)
+        # TODO: as the 'HEAD' reference is working tree specific, it
+        # should actually not be a part of RefsContainer
+        if name == 'HEAD':
+            return os.path.join(self.worktree_path, name)
+        else:
+            return os.path.join(self.path, name)
 
     def get_packed_refs(self):
         """Get contents of the packed-refs file.

dulwich/repo.py

@@ -89,6 +89,9 @@ REFSDIR = 'refs'
 REFSDIR_TAGS = 'tags'
 REFSDIR_HEADS = 'heads'
 INDEX_FILENAME = "index"
+COMMONDIR = 'commondir'
+GITDIR = 'gitdir'
+WORKTREES = 'worktrees'
 
 BASE_DIRECTORIES = [
     ["branches"],
@@ -99,6 +102,8 @@ BASE_DIRECTORIES = [
     ["info"]
     ]
 
+DEFAULT_REF = b'refs/heads/master'
+
 
 def parse_graftpoints(graftpoints):
     """Convert a list of graftpoints into a dict
@@ -676,18 +681,28 @@ class Repo(BaseRepo):
             raise NotGitRepository(
                 "No git repository was found at %(path)s" % dict(path=root)
             )
+        commondir = self.get_named_file(COMMONDIR)
+        if commondir is not None:
+            with commondir:
+                self._commondir = os.path.join(
+                    self.controldir(),
+                    commondir.read().rstrip(b"\r\n").decode(sys.getfilesystemencoding()))
+        else:
+            self._commondir = self._controldir
         self.path = root
-        object_store = DiskObjectStore(os.path.join(self.controldir(),
-                                                    OBJECTDIR))
-        refs = DiskRefsContainer(self.controldir())
+        object_store = DiskObjectStore(
+            os.path.join(self.commondir(), OBJECTDIR))
+        refs = DiskRefsContainer(self.commondir(), self._controldir)
         BaseRepo.__init__(self, object_store, refs)
 
         self._graftpoints = {}
-        graft_file = self.get_named_file(os.path.join("info", "grafts"))
+        graft_file = self.get_named_file(os.path.join("info", "grafts"),
+                                         basedir=self.commondir())
         if graft_file:
             with graft_file:
                 self._graftpoints.update(parse_graftpoints(graft_file))
-        graft_file = self.get_named_file("shallow")
+        graft_file = self.get_named_file("shallow",
+                                         basedir=self.commondir())
         if graft_file:
             with graft_file:
                 self._graftpoints.update(parse_graftpoints(graft_file))
@@ -720,6 +735,16 @@ class Repo(BaseRepo):
         """Return the path of the control directory."""
         return self._controldir
 
+    def commondir(self):
+        """Return the path of the common directory.
+
+        For a main working tree, it is identical to `controldir()`.
+
+        For a linked working tree, it is the control directory of the
+        main working tree."""
+
+        return self._commondir
+
     def _put_named_file(self, path, contents):
         """Write a file to the control dir with the given name and contents.
 
@@ -730,7 +755,7 @@ class Repo(BaseRepo):
         with GitFile(os.path.join(self.controldir(), path), 'wb') as f:
             f.write(contents)
 
-    def get_named_file(self, path):
+    def get_named_file(self, path, basedir=None):
         """Get a file from the control dir with a specific name.
 
         Although the filename should be interpreted as a filename relative to
@@ -738,13 +763,16 @@ class Repo(BaseRepo):
         pointing to a file in that location.
 
         :param path: The path to the file, relative to the control dir.
+        :param basedir: Optional argument that specifies an alternative to the control dir.
         :return: An open file object, or None if the file does not exist.
         """
         # TODO(dborowitz): sanitize filenames, since this is used directly by
         # the dumb web serving code.
+        if basedir is None:
+            basedir = self.controldir()
         path = path.lstrip(os.path.sep)
         try:
-            return open(os.path.join(self.controldir(), path), 'rb')
+            return open(os.path.join(basedir, path), 'rb')
         except (IOError, OSError) as e:
             if e.errno == errno.ENOENT:
                 return None
@@ -827,9 +855,7 @@ class Repo(BaseRepo):
         target.refs.import_refs(
             b'refs/tags', self.refs.as_dict(b'refs/tags'))
         try:
-            target.refs.add_if_new(
-                b'refs/heads/master',
-                self.refs[b'refs/heads/master'])
+            target.refs.add_if_new(DEFAULT_REF, self.refs[DEFAULT_REF])
         except KeyError:
             pass
 
@@ -914,7 +940,7 @@ class Repo(BaseRepo):
             os.mkdir(os.path.join(path, *d))
         DiskObjectStore.init(os.path.join(path, OBJECTDIR))
         ret = cls(path)
-        ret.refs.set_symbolic_ref(b'HEAD', b"refs/heads/master")
+        ret.refs.set_symbolic_ref(b'HEAD', DEFAULT_REF)
         ret._init_files(bare)
         return ret
 
@@ -933,6 +959,47 @@ class Repo(BaseRepo):
         cls._init_maybe_bare(controldir, False)
         return cls(path)
 
+    @classmethod
+    def _init_new_working_directory(cls, path, main_repo, identifier=None, mkdir=False):
+        """Create a new working directory linked to a repository.
+
+        :param path: Path in which to create the working tree.
+        :param main_repo: Main repository to reference
+        :param identifier: Worktree identifier
+        :param mkdir: Whether to create the directory
+        :return: `Repo` instance
+        """
+        if mkdir:
+            os.mkdir(path)
+        if identifier is None:
+            identifier = os.path.basename(path)
+        main_worktreesdir = os.path.join(main_repo.controldir(), WORKTREES)
+        worktree_controldir = os.path.join(main_worktreesdir, identifier)
+        gitdirfile = os.path.join(path, CONTROLDIR)
+        with open(gitdirfile, 'wb') as f:
+            f.write(b'gitdir: ' +
+                    worktree_controldir.encode(sys.getfilesystemencoding()) +
+                    b'\n')
+        try:
+            os.mkdir(main_worktreesdir)
+        except OSError as e:
+            if e.errno != errno.EEXIST:
+                raise
+        try:
+            os.mkdir(worktree_controldir)
+        except OSError as e:
+            if e.errno != errno.EEXIST:
+                raise
+        with open(os.path.join(worktree_controldir, GITDIR), 'wb') as f:
+            f.write(gitdirfile.encode(sys.getfilesystemencoding()) + b'\n')
+        with open(os.path.join(worktree_controldir, COMMONDIR), 'wb') as f:
+            f.write(b'../..\n')
+        with open(os.path.join(worktree_controldir, 'HEAD'), 'wb') as f:
+            f.write(main_repo.head() + b'\n')
+        r = cls(path)
+        r.reset_index()
+        return r
+
     @classmethod
     def init_bare(cls, path):
         """Create a new bare repository.

dulwich/tests/compat/test_client.py

@@ -66,9 +66,9 @@ from dulwich.tests.compat.utils import (
     CompatTestCase,
     check_for_daemon,
     import_repo_to_dir,
+    rmtree_ro,
     run_git_or_fail,
     _DEFAULT_GIT,
-    rmtree_ro,
     )
 
 

dulwich/tests/compat/test_repository.py

@@ -24,15 +24,17 @@
 from io import BytesIO
 from itertools import chain
 import os
+import tempfile
 
 from dulwich.objects import (
     hex_to_sha,
     )
 from dulwich.repo import (
     check_ref_format,
+    Repo,
     )
-
 from dulwich.tests.compat.utils import (
+    rmtree_ro,
     run_git_or_fail,
     CompatTestCase,
     )
@@ -119,3 +121,89 @@ class ObjectStoreTestCase(CompatTestCase):
     def test_all_objects(self):
         expected_shas = self._get_all_shas()
         self.assertShasMatch(expected_shas, iter(self._repo.object_store))
+
+
+class WorkingTreeTestCase(ObjectStoreTestCase):
+    """Test for compatibility with git-worktree."""
+
+    min_git_version = (2, 5, 0)
+
+    def create_new_worktree(self, repo_dir, branch):
+        """Create a new worktree using git-worktree.
+
+        :param repo_dir: The directory of the main working tree.
+        :param branch: The branch or commit to checkout in the new worktree.
+
+        :returns: The path to the new working tree.
+        """
+        temp_dir = tempfile.mkdtemp()
+        run_git_or_fail(['worktree', 'add', temp_dir, branch],
+                        cwd=repo_dir)
+        self.addCleanup(rmtree_ro, temp_dir)
+        return temp_dir
+
+    def setUp(self):
+        super(WorkingTreeTestCase, self).setUp()
+        self._worktree_path = self.create_new_worktree(self._repo.path, 'branch')
+        self._worktree_repo = Repo(self._worktree_path)
+        self.addCleanup(self._worktree_repo.close)
+        self._mainworktree_repo = self._repo
+        self._number_of_working_tree = 2
+        self._repo = self._worktree_repo
+
+    def test_refs(self):
+        super(WorkingTreeTestCase, self).test_refs()
+        self.assertEqual(self._mainworktree_repo.refs.allkeys(),
+                         self._repo.refs.allkeys())
+
+    def test_head_equality(self):
+        self.assertNotEqual(self._repo.refs[b'HEAD'],
+                            self._mainworktree_repo.refs[b'HEAD'])
+
+    def test_bare(self):
+        self.assertFalse(self._repo.bare)
+        self.assertTrue(os.path.isfile(os.path.join(self._repo.path, '.git')))
+
+    def _parse_worktree_list(self, output):
+        worktrees = []
+        for line in BytesIO(output):
+            fields = line.rstrip(b'\n').split()
+            worktrees.append(tuple(f.decode() for f in fields))
+        return worktrees
+
+    def test_git_worktree_list(self):
+        output = run_git_or_fail(['worktree', 'list'], cwd=self._repo.path)
+        worktrees = self._parse_worktree_list(output)
+        self.assertEqual(len(worktrees), self._number_of_working_tree)
+        self.assertEqual(worktrees[0][1], '(bare)')
+        self.assertEqual(worktrees[0][0], self._mainworktree_repo.path)
+
+        output = run_git_or_fail(['worktree', 'list'], cwd=self._mainworktree_repo.path)
+        worktrees = self._parse_worktree_list(output)
+        self.assertEqual(len(worktrees), self._number_of_working_tree)
+        self.assertEqual(worktrees[0][1], '(bare)')
+        self.assertEqual(worktrees[0][0], self._mainworktree_repo.path)
+
+
+class InitNewWorkingDirectoryTestCase(WorkingTreeTestCase):
+    """Test compatibility of Repo.init_new_working_directory."""
+
+    min_git_version = (2, 5, 0)
+
+    def setUp(self):
+        super(InitNewWorkingDirectoryTestCase, self).setUp()
+        self._other_worktree = self._repo
+        worktree_repo_path = tempfile.mkdtemp()
+        self.addCleanup(rmtree_ro, worktree_repo_path)
+        self._repo = Repo._init_new_working_directory(
+            worktree_repo_path, self._mainworktree_repo)
+        self.addCleanup(self._repo.close)
+        self._number_of_working_tree = 3
+
+    def test_head_equality(self):
+        self.assertEqual(self._repo.refs[b'HEAD'],
+                         self._mainworktree_repo.refs[b'HEAD'])
+
+    def test_bare(self):
+        self.assertFalse(self._repo.bare)
+        self.assertTrue(os.path.isfile(os.path.join(self._repo.path, '.git')))

dulwich/tests/test_client.py

@@ -112,7 +112,8 @@ class GitClientTests(TestCase):
         def check_heads(heads):
             self.assertIs(heads, None)
             return []
-        self.client.fetch_pack(b'/', check_heads, None, None)
+        ret = self.client.fetch_pack(b'/', check_heads, None, None)
+        self.assertIs(None, ret)
 
     def test_fetch_pack_ignores_magic_ref(self):
         self.rin.write(
@@ -124,7 +125,8 @@ class GitClientTests(TestCase):
         def check_heads(heads):
             self.assertEquals({}, heads)
             return []
-        self.client.fetch_pack(b'bla', check_heads, None, None, None)
+        ret = self.client.fetch_pack(b'bla', check_heads, None, None, None)
+        self.assertIs(None, ret)
         self.assertEqual(self.rout.getvalue(), b'0000')
 
     def test_fetch_pack_none(self):
@@ -378,6 +380,22 @@ class TestGetTransportAndPath(TestCase):
         self.assertEqual(1234, c.port)
         self.assertEqual('bar/baz', path)
 
+    def test_username_and_port_explicit_unknown_scheme(self):
+        c, path = get_transport_and_path(
+            'unknown://git@server:7999/dply/stuff.git')
+        self.assertTrue(isinstance(c, SSHGitClient))
+        self.assertEqual('unknown', c.host)
+        self.assertEqual('//git@server:7999/dply/stuff.git', path)
+
+    def test_username_and_port_explicit(self):
+        c, path = get_transport_and_path(
+            'ssh://git@server:7999/dply/stuff.git')
+        self.assertTrue(isinstance(c, SSHGitClient))
+        self.assertEqual('git', c.username)
+        self.assertEqual('server', c.host)
+        self.assertEqual(7999, c.port)
+        self.assertEqual('dply/stuff.git', path)
+
     def test_ssh_abspath_explicit(self):
         c, path = get_transport_and_path('git+ssh://foo.com//bar/baz')
         self.assertTrue(isinstance(c, SSHGitClient))
@@ -465,6 +483,26 @@ class TestGetTransportAndPath(TestCase):
         self.assertTrue(isinstance(c, HttpGitClient))
         self.assertEqual('/jelmer/dulwich', path)
 
+    def test_http_auth(self):
+        url = 'https://user:passwd@github.com/jelmer/dulwich'
+
+        c, path = get_transport_and_path(url)
+
+        self.assertTrue(isinstance(c, HttpGitClient))
+        self.assertEqual('/jelmer/dulwich', path)
+        self.assertEqual('user', c._username)
+        self.assertEqual('passwd', c._password)
+
+    def test_http_no_auth(self):
+        url = 'https://github.com/jelmer/dulwich'
+
+        c, path = get_transport_and_path(url)
+
+        self.assertTrue(isinstance(c, HttpGitClient))
+        self.assertEqual('/jelmer/dulwich', path)
+        self.assertIs(None, c._username)
+        self.assertIs(None, c._password)
+
 
 class TestGetTransportAndPathFromUrl(TestCase):
 
@@ -677,8 +715,14 @@ class LocalGitClientTests(TestCase):
         self.addCleanup(tear_down_repo, s)
         out = BytesIO()
         walker = {}
-        c.fetch_pack(s.path, lambda heads: [], graph_walker=walker,
+        ret = c.fetch_pack(s.path, lambda heads: [], graph_walker=walker,
             pack_data=out.write)
+        self.assertEqual({
+            b'HEAD': b'a90fa2d900a17e99b433217e988c4eb4a2e9a097',
+            b'refs/heads/master': b'a90fa2d900a17e99b433217e988c4eb4a2e9a097',
+            b'refs/tags/mytag': b'28237f4dc30d0d462658d6b937b08a0f0b6ef55a',
+            b'refs/tags/mytag-packed': b'b0931cadc54336e78a1d980420e3268903b57a50'
+            }, ret)
         self.assertEqual(b"PACK\x00\x00\x00\x02\x00\x00\x00\x00\x02\x9d\x08"
             b"\x82;\xd8\xa8\xea\xb5\x10\xadj\xc7\\\x82<\xfd>\xd3\x1e", out.getvalue())
 
@@ -745,6 +789,37 @@ class HttpGitClientTests(TestCase):
         url = c.get_url(path)
         self.assertEqual('https://github.com/jelmer/dulwich', url)
 
+    def test_get_url_with_username_and_passwd(self):
+        base_url = 'https://github.com/jelmer/dulwich'
+        path = '/jelmer/dulwich'
+        c = HttpGitClient(base_url, username='USERNAME', password='PASSWD')
+
+        url = c.get_url(path)
+        self.assertEqual('https://github.com/jelmer/dulwich', url)
+
+    def test_init_username_passwd_set(self):
+        url = 'https://github.com/jelmer/dulwich'
+
+        c = HttpGitClient(url, config=None, username='user', password='passwd')
+        self.assertEqual('user', c._username)
+        self.assertEqual('passwd', c._password)
+        [pw_handler] = [
+            h for h in c.opener.handlers if getattr(h, 'passwd', None) is not None]
+        self.assertEqual(
+            ('user', 'passwd'),
+            pw_handler.passwd.find_user_password(
+                None, 'https://github.com/jelmer/dulwich'))
+
+    def test_init_no_username_passwd(self):
+        url = 'https://github.com/jelmer/dulwich'
+
+        c = HttpGitClient(url, config=None)
+        self.assertIs(None, c._username)
+        self.assertIs(None, c._password)
+        pw_handler = [
+            h for h in c.opener.handlers if getattr(h, 'passwd', None) is not None]
+        self.assertEqual(0, len(pw_handler))
+
 
 class TCPGitClientTests(TestCase):
 

dulwich/tests/test_objects.py

@@ -136,6 +136,22 @@ class BlobReadTests(TestCase):
         b = Blob.from_string(string)
         self.assertEqual([string], b.chunked)
 
+    def test_splitlines(self):
+        for case in [
+            [],
+            [b'foo\nbar\n'],
+            [b'bl\na', b'blie'],
+            [b'bl\na', b'blie', b'bloe\n'],
+            [b'', b'bl\na', b'blie', b'bloe\n'],
+            [b'', b'', b'', b'bla\n'],
+            [b'', b'', b'', b'bla\n', b''],
+            [b'bl', b'', b'a\naaa'],
+            [b'a\naaa', b'a'],
+            ]:
+            b = Blob()
+            b.chunked = case
+            self.assertEqual(b.data.splitlines(True), b.splitlines())
+
     def test_set_chunks(self):
         b = Blob()
         b.chunked = [b'te', b'st', b' 5\n']
@@ -302,6 +318,16 @@ class CommitSerializationTests(TestCase):
         c1.set_raw_string(c.as_raw_string())
         self.assertEqual(30, c1.commit_time)
 
+    def test_full_tree(self):
+        c = self.make_commit(commit_time=30)
+        t = Tree()
+        t.add(b'data-x', 0o644, Blob().id)
+        c.tree = t
+        c1 = Commit()
+        c1.set_raw_string(c.as_raw_string())
+        self.assertEqual(t.id, c1.tree)
+        self.assertEqual(c.as_raw_string(), c1.as_raw_string())
+
     def test_raw_length(self):
         c = self.make_commit()
         self.assertEqual(len(c.as_raw_string()), c.raw_length())

dulwich/tests/test_porcelain.py

@@ -123,7 +123,7 @@ class CloneTests(PorcelainTestCase):
         self.assertEqual(r.path, target_path)
         target_repo = Repo(target_path)
         self.assertEqual(target_repo.head(), c3.id)
-        self.assertEquals(c3.id, target_repo.refs[b'refs/tags/foo'])
+        self.assertEqual(c3.id, target_repo.refs[b'refs/tags/foo'])
         self.assertTrue(b'f1' not in os.listdir(target_path))
         self.assertTrue(b'f2' not in os.listdir(target_path))
 
@@ -444,6 +444,30 @@ class ResetTests(PorcelainTestCase):
 
         self.assertEqual([], changes)
 
+    def test_hard_commit(self):
+        with open(os.path.join(self.repo.path, 'foo'), 'w') as f:
+            f.write("BAR")
+        porcelain.add(self.repo.path, paths=["foo"])
+        sha = porcelain.commit(self.repo.path, message=b"Some message",
+                committer=b"Jane <jane@example.com>",
+                author=b"John <john@example.com>")
+
+        with open(os.path.join(self.repo.path, 'foo'), 'wb') as f:
+            f.write(b"BAZ")
+        porcelain.add(self.repo.path, paths=["foo"])
+        porcelain.commit(self.repo.path, message=b"Some other message",
+                committer=b"Jane <jane@example.com>",
+                author=b"John <john@example.com>")
+
+        porcelain.reset(self.repo, "hard", sha)
+
+        index = self.repo.open_index()
+        changes = list(tree_changes(self.repo,
+                       index.commit(self.repo.object_store),
+                       self.repo[sha].tree))
+
+        self.assertEqual([], changes)
+
 
 class PushTests(PorcelainTestCase):
 

dulwich/tests/test_repository.py

@@ -521,6 +521,32 @@ exit 1
             check(nonbare)
             check(bare)
 
+    def test_working_tree(self):
+        temp_dir = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, temp_dir)
+        worktree_temp_dir = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, worktree_temp_dir)
+        r = Repo.init(temp_dir)
+        root_sha = r.do_commit(
+                b'empty commit',
+                committer=b'Test Committer <test@nodomain.com>',
+                author=b'Test Author <test@nodomain.com>',
+                commit_timestamp=12345, commit_timezone=0,
+                author_timestamp=12345, author_timezone=0)
+        r.refs[b'refs/heads/master'] = root_sha
+        w = Repo._init_new_working_directory(worktree_temp_dir, r)
+        new_sha = w.do_commit(
+                b'new commit',
+                committer=b'Test Committer <test@nodomain.com>',
+                author=b'Test Author <test@nodomain.com>',
+                commit_timestamp=12345, commit_timezone=0,
+                author_timestamp=12345, author_timezone=0)
+        w.refs[b'HEAD'] = new_sha
+        self.assertEqual(os.path.abspath(r.controldir()),
+                         os.path.abspath(w.commondir()))
+        self.assertEqual(r.refs.keys(), w.refs.keys())
+        self.assertNotEqual(r.head(), w.head())
+
 
 class BuildRepoRootTests(TestCase):
     """Tests that build on-disk repos from scratch.

examples/diff.py

@@ -10,10 +10,11 @@ from dulwich.patch import write_tree_diff
 import sys
 
 repo_path = "."
-commit_id = "a6602654997420bcfd0bee2a0563d9416afe34b4"
+commit_id = b"a6602654997420bcfd0bee2a0563d9416afe34b4"
 
 r = Repo(repo_path)
 
 commit = r[commit_id]
 parent_commit = r[commit.parents[0]]
-write_tree_diff(sys.stdout, r.object_store, parent_commit.tree, commit.tree)
+outstream = getattr(sys.stdout, 'buffer', sys.stdout)
+write_tree_diff(outstream, r.object_store, parent_commit.tree, commit.tree)

examples/latest_change.py

@@ -11,7 +11,9 @@ if len(sys.argv) < 2:
 
 r = Repo(".")
 
-w = r.get_walker(paths=[sys.argv[1]], max_entries=1)
+path = sys.argv[1].encode('utf-8')
+
+w = r.get_walker(paths=[path], max_entries=1)
 try:
     c = next(iter(w)).commit
 except StopIteration:

setup.py

@@ -9,7 +9,7 @@ except ImportError:
     from distutils.core import setup, Extension
 from distutils.core import Distribution
 
-dulwich_version_string = '0.15.0'
+dulwich_version_string = '0.16.0'
 
 include_dirs = []
 # Windows MSVC support