Browse Source

Merge branch 'develop' (release/0.5)

tags/v0.5.1
Ben Kurtovic 6 years ago
parent
commit
31d7486224
8 changed files with 160 additions and 88 deletions
  1. +16
    -1
      CHANGELOG
  2. +1
    -1
      LICENSE
  3. +16
    -12
      README.md
  4. +3
    -3
      gitup/__init__.py
  5. +9
    -0
      gitup/__main__.py
  6. +14
    -9
      gitup/script.py
  7. +93
    -56
      gitup/update.py
  8. +8
    -6
      setup.py

+ 16
- 1
CHANGELOG View File

@@ -1,3 +1,18 @@
v0.5 (released August 28, 2018):

- Added a `--depth` flag to control recursion depth when searching for
repositories inside of subdirectories. For example:
- `--depth 0` will never recurse into subdirectories; the provided paths must
be repositories by themselves.
- `--depth 1` will descend one level to look for repositories. This is the
old behavior.
- `--depth 3` will look three levels deep. This is the new default.
- `--depth -1` will recurse indefinitely. This is not recommended.
- Allow gitup to be run directly as a Python module (python -m gitup).
- Fixed an error when updating branches if the upstream is completely unrelated
from the local branch (no common ancestor).
- Fixed error message when fetching from a remote fails.

v0.4.1 (released December 13, 2017):

- Bump dependencies to deal with newer versions of Git.
@@ -13,7 +28,7 @@ v0.4 (released January 17, 2017):
- Cleaned up the bookmark file format, fixing a related Windows bug. The script
will automatically migrate to the new one.
- Fixed a bug related to Python 3 compatibility.
- Fixed unicode support.
- Fixed Unicode support.

v0.3 (released June 7, 2015):



+ 1
- 1
LICENSE View File

@@ -1,4 +1,4 @@
Copyright (C) 2011-2017 Ben Kurtovic <ben.kurtovic@gmail.com>
Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal


+ 16
- 12
README.md View File

@@ -1,9 +1,9 @@
__gitup__ (the _git-repo-updater_)

gitup is a tool designed to update a large number of git repositories at once.
It is smart enough to handle multiple remotes, branches, dirty working
directories, and more, hopefully providing a great way to get everything
up-to-date for short periods of internet access between long periods of none.
gitup is a tool for updating multiple git repositories at once. It is smart
enough to handle several remotes, dirty working directories, diverged local
branches, detached HEADs, and more. It was originally created to manage a large
collection of projects and deal with sporadic internet access.

gitup should work on OS X, Linux, and Windows. You should have the latest
version of git and either Python 2.7 or Python 3 installed.
@@ -25,7 +25,7 @@ Then, to install for everyone:

sudo python setup.py install

...or for just yourself (make sure you have `~/.local/bin` in your PATH):
or for just yourself (make sure you have `~/.local/bin` in your PATH):

python setup.py install --user

@@ -53,7 +53,7 @@ Additionally, you can just type:

to automatically update all git repositories in that directory.

To add a bookmark (or bookmarks), either of these will work:
To add bookmarks, either of these will work:

gitup --add ~/repos/foo ~/repos/bar ~/repos/baz
gitup --add ~/repos
@@ -81,21 +81,25 @@ Update all git repositories in your current directory:

gitup .

You can control how deep gitup will look for repositories in a given directory,
if that directory is not a git repo by itself, with the `--depth` (or `-t`)
option. `--depth 0` will disable recursion entirely, meaning the provided paths
must be repos by themselves. `--depth 1` will descend one level (this is the
old behavior from pre-0.5 gitup). `--depth -1` will recurse indefinitely,
which is not recommended. The default is `--depth 3`.

By default, gitup will fetch all remotes in a repository. Pass `--current-only`
(or `-c`) to make it fetch _only_ the remote tracked by the current branch.
(or `-c`) to make it fetch only the remote tracked by the current branch.

Also by default, gitup will try to fast-forward all branches that have
upstreams configured. It will always skip branches where this is not possible
(e.g. dirty working directory or a merge/rebase is required). Pass
`--fetch-only` (or `-f`) to only fetch remotes.
`--fetch-only` (or `-f`) to skip this step and only fetch remotes.

After fetching, gitup will _keep_ remote-tracking branches that no longer exist
upstream. Pass `--prune` (or `-p`) to delete them, or set `fetch.prune` or
`remote.<name>.prune` in git config to do this by default.
`remote.<name>.prune` in your git config to do this by default.

For a full list of all command arguments and abbreviations:

gitup --help

Finally, all paths can be either absolute (e.g. `/path/to/repo`) or relative
(e.g. `../my/repo`).

+ 3
- 3
gitup/__init__.py View File

@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2011-2017 Ben Kurtovic <ben.kurtovic@gmail.com>
# Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>
# Released under the terms of the MIT License. See LICENSE for details.

"""
@@ -8,7 +8,7 @@ gitup: the git repository updater
"""

__author__ = "Ben Kurtovic"
__copyright__ = "Copyright (C) 2011-2017 Ben Kurtovic"
__copyright__ = "Copyright (C) 2011-2018 Ben Kurtovic"
__license__ = "MIT License"
__version__ = "0.4.1"
__version__ = "0.5"
__email__ = "ben.kurtovic@gmail.com"

+ 9
- 0
gitup/__main__.py View File

@@ -0,0 +1,9 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>
# Released under the terms of the MIT License. See LICENSE for details.

from .script import run

if __name__ == "__main__":
run()

+ 14
- 9
gitup/script.py View File

@@ -1,12 +1,13 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2011-2016 Ben Kurtovic <ben.kurtovic@gmail.com>
# Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>
# Released under the terms of the MIT License. See LICENSE for details.

from __future__ import print_function

import argparse
import os
import platform
import sys

from colorama import init as color_init, Fore, Style
@@ -39,12 +40,16 @@ def main():

group_u.add_argument(
'directories_to_update', nargs="*", metavar="path", type=_decode,
help="""update all repositories in this directory (or the directory
itself, if it is a repo)""")
help="""update this repository, or all repositories it contains
(if not a repo directly)""")
group_u.add_argument(
'-u', '--update', action="store_true", help="""update all bookmarks
(default behavior when called without arguments)""")
group_u.add_argument(
'-t', '--depth', dest="max_depth", metavar="n", type=int, default=3,
help="""max recursion depth when searching for repos in subdirectories
(default: 3; use 0 for no recursion, or -1 for unlimited)""")
group_u.add_argument(
'-c', '--current-only', action="store_true", help="""only fetch the
remote tracked by the current branch instead of all remotes""")
group_u.add_argument(
@@ -80,7 +85,8 @@ def main():
'-h', '--help', action="help", help="show this help message and exit")
group_m.add_argument(
'-v', '--version', action="version",
version="gitup " + __version__)
version="gitup {0} (Python {1})".format(
__version__, platform.python_version()))

# TODO: deprecated arguments, for removal in v1.0:
parser.add_argument(
@@ -90,7 +96,6 @@ def main():

color_init(autoreset=True)
args = parser.parse_args()
update_args = args.current_only, args.fetch_only, args.prune

print(Style.BRIGHT + "gitup" + Style.RESET_ALL + ": the git-repo-updater")
print()
@@ -121,15 +126,15 @@ def main():

if args.command:
if args.directories_to_update:
run_command(args.directories_to_update, args.command)
run_command(args.directories_to_update, args)
if args.update or not args.directories_to_update:
run_command(get_bookmarks(args.bookmark_file), args.command)
run_command(get_bookmarks(args.bookmark_file), args)
else:
if args.directories_to_update:
update_directories(args.directories_to_update, update_args)
update_directories(args.directories_to_update, args)
acted = True
if args.update or not acted:
update_bookmarks(get_bookmarks(args.bookmark_file), update_args)
update_bookmarks(get_bookmarks(args.bookmark_file), args)

def run():
"""Thin wrapper for main() that catches KeyboardInterrupts."""


+ 93
- 56
gitup/update.py View File

@@ -1,12 +1,14 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2011-2016 Ben Kurtovic <ben.kurtovic@gmail.com>
# Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>
# Released under the terms of the MIT License. See LICENSE for details.

from __future__ import print_function

from glob import glob
import os
import pipes
import re
import shlex

from colorama import Fore, Style
@@ -77,8 +79,14 @@ def _fetch_remotes(remotes, prune):
try:
results = remote.fetch(progress=_ProgressMonitor(), prune=prune)
except exc.GitCommandError as err:
msg = err.command[0].replace("Error when fetching: ", "")
if not msg.endswith("."):
# We should have to do this ourselves, but GitPython doesn't give
# us a sensible way to get the raw stderr...
msg = re.sub(r"\s+", " ", err.stderr).strip()
msg = re.sub(r"^stderr: *'(fatal: *)?", "", msg).strip("'")
if not msg:
command = " ".join(pipes.quote(arg) for arg in err.command)
msg = "{0} failed with status {1}.".format(command, err.status)
elif not msg.endswith("."):
msg += "."
print(":", RED + "error:", msg)
return
@@ -115,7 +123,12 @@ def _update_branch(repo, branch, is_active=False):
print(YELLOW + "skipped:", "upstream does not exist.")
return

base = repo.git.merge_base(branch.commit, upstream.commit)
try:
base = repo.git.merge_base(branch.commit, upstream.commit)
except exc.GitCommandError as err:
print(YELLOW + "skipped:", "can't find merge base with upstream.")
return

if repo.commit(base) == upstream.commit:
print(BLUE + "up to date", end=".\n")
return
@@ -140,23 +153,23 @@ def _update_branch(repo, branch, is_active=False):
repo.git.branch(branch.name, upstream.name, force=True)
print(GREEN + "done", end=".\n")

def _update_repository(repo, current_only, fetch_only, prune):
def _update_repository(repo, repo_name, args):
"""Update a single git repository by fetching remotes and rebasing/merging.

The specific actions depend on the arguments given. We will fetch all
remotes if *current_only* is ``False``, or only the remote tracked by the
current branch if ``True``. If *fetch_only* is ``False``, we will also
update all fast-forwardable branches that are tracking valid upstreams.
If *prune* is ``True``, remote-tracking branches that no longer exist on
their remote after fetching will be deleted.
remotes if *args.current_only* is ``False``, or only the remote tracked by
the current branch if ``True``. If *args.fetch_only* is ``False``, we will
also update all fast-forwardable branches that are tracking valid
upstreams. If *args.prune* is ``True``, remote-tracking branches that no
longer exist on their remote after fetching will be deleted.
"""
print(INDENT1, BOLD + os.path.split(repo.working_dir)[1] + ":")
print(INDENT1, BOLD + repo_name + ":")

try:
active = repo.active_branch
except TypeError: # Happens when HEAD is detached
active = None
if current_only:
if args.current_only:
if not active:
print(INDENT2, ERROR,
"--current-only doesn't make sense with a detached HEAD.")
@@ -172,17 +185,17 @@ def _update_repository(repo, current_only, fetch_only, prune):
if not remotes:
print(INDENT2, ERROR, "no remotes configured to fetch.")
return
_fetch_remotes(remotes, prune)
_fetch_remotes(remotes, args.prune)

if not fetch_only:
if not args.fetch_only:
for branch in sorted(repo.heads, key=lambda b: b.name):
_update_branch(repo, branch, branch == active)

def _run_command(repo, command):
def _run_command(repo, repo_name, args):
"""Run an arbitrary shell command on the given repository."""
print(INDENT1, BOLD + os.path.split(repo.working_dir)[1] + ":")
print(INDENT1, BOLD + repo_name + ":")

cmd = shlex.split(command)
cmd = shlex.split(args.command)
try:
out = repo.git.execute(
cmd, with_extended_output=True, with_exceptions=False)
@@ -193,24 +206,7 @@ def _run_command(repo, command):
for line in out[1].splitlines() + out[2].splitlines():
print(INDENT2, line)

def _dispatch_multi(base, paths, callback, *args):
"""Apply the callback to all git repos in the list of paths."""
valid = []
for path in paths:
try:
Repo(path)
except (exc.InvalidGitRepositoryError, exc.NoSuchPathError):
continue
valid.append(path)

base = os.path.abspath(base)
suffix = "" if len(valid) == 1 else "s"
print(BOLD + base, "({0} repo{1}):".format(len(valid), suffix))

for path in sorted(valid, key=os.path.basename):
callback(Repo(path), *args)

def _dispatch(path, callback, *args):
def _dispatch(base_path, callback, args):
"""Apply a callback function on each valid repo in the given path.

Determine whether the directory is a git repo on its own, a directory of
@@ -220,40 +216,81 @@ def _dispatch(path, callback, *args):

The given args are passed directly to the callback function after the repo.
"""
path = os.path.expanduser(path)
def _collect(paths, max_depth):
"""Return all valid repo paths in the given paths, recursively."""
if max_depth == 0:
return []

valid = []
for path in paths:
try:
Repo(path)
valid.append(path)
except exc.InvalidGitRepositoryError:
if not os.path.isdir(path):
continue
children = [os.path.join(path, it) for it in os.listdir(path)]
valid += _collect(children, max_depth - 1)
except exc.NoSuchPathError:
continue
return valid

def _get_basename(base, path):
"""Return a reasonable name for a repo path in the given base."""
if path.startswith(base + os.path.sep):
return path.split(base + os.path.sep, 1)[1]
prefix = os.path.commonprefix([base, path])
while not base.startswith(prefix + os.path.sep):
old = prefix
prefix = os.path.split(prefix)[0]
if prefix == old:
break # Prevent infinite loop, but should be almost impossible
return path.split(prefix + os.path.sep, 1)[1]

base = os.path.expanduser(base_path)
max_depth = args.max_depth
if max_depth >= 0:
max_depth += 1

try:
repo = Repo(path)
Repo(base)
valid = [base]
except exc.NoSuchPathError:
paths = glob(path)
if paths:
_dispatch_multi(path, paths, callback, *args)
else:
print(ERROR, BOLD + path, "doesn't exist!")
paths = glob(base)
if not paths:
print(ERROR, BOLD + base, "doesn't exist!")
return
valid = _collect(paths, max_depth)
except exc.InvalidGitRepositoryError:
if os.path.isdir(path):
paths = [os.path.join(path, item) for item in os.listdir(path)]
_dispatch_multi(path, paths, callback, *args)
else:
print(ERROR, BOLD + path, "isn't a repository!")
else:
print(BOLD + repo.working_dir, "(1 repo):")
callback(repo, *args)
if not os.path.isdir(base) or args.max_depth == 0:
print(ERROR, BOLD + base, "isn't a repository!")
return
valid = _collect([base], max_depth)

base = os.path.abspath(base)
suffix = "" if len(valid) == 1 else "s"
print(BOLD + base, "({0} repo{1}):".format(len(valid), suffix))

valid = [os.path.abspath(path) for path in valid]
paths = [(_get_basename(base, path), path) for path in valid]
for name, path in sorted(paths):
callback(Repo(path), name, args)

def update_bookmarks(bookmarks, update_args):
def update_bookmarks(bookmarks, args):
"""Loop through and update all bookmarks."""
if not bookmarks:
print("You don't have any bookmarks configured! Get help with 'gitup -h'.")
return

for path in bookmarks:
_dispatch(path, _update_repository, *update_args)
_dispatch(path, _update_repository, args)

def update_directories(paths, update_args):
def update_directories(paths, args):
"""Update a list of directories supplied by command arguments."""
for path in paths:
_dispatch(path, _update_repository, *update_args)
_dispatch(path, _update_repository, args)

def run_command(paths, command):
def run_command(paths, args):
"""Run an arbitrary shell command on all repos."""
for path in paths:
_dispatch(path, _run_command, command)
_dispatch(path, _run_command, args)

+ 8
- 6
setup.py View File

@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2011-2016 Ben Kurtovic <ben.kurtovic@gmail.com>
# Copyright (C) 2011-2018 Ben Kurtovic <ben.kurtovic@gmail.com>
# Released under the terms of the MIT License. See LICENSE for details.

import sys
@@ -8,7 +8,7 @@ import sys
from setuptools import setup, find_packages

if sys.hexversion < 0x02070000:
exit("Please upgrade to Python 2.7 or greater: <http://python.org/>.")
exit("Please upgrade to Python 2.7 or greater: <https://www.python.org/>.")

from gitup import __version__

@@ -23,18 +23,19 @@ setup(
version = __version__,
author = "Ben Kurtovic",
author_email = "ben.kurtovic@gmail.com",
description = "Easily pull to multiple git repositories at once",
description = "Easily update multiple git repositories at once",
long_description = long_desc,
license = "MIT License",
keywords = "git repository pull update",
url = "http://github.com/earwig/git-repo-updater",
url = "https://github.com/earwig/git-repo-updater",
classifiers = [
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Natural Language :: English",
"Operating System :: MacOS :: MacOS X",
"Operating System :: POSIX :: Linux",
"Operating System :: POSIX",
"Operating System :: Microsoft :: Windows",
"Programming Language :: Python",
"Programming Language :: Python :: 2.7",
@@ -44,6 +45,7 @@ setup(
"Programming Language :: Python :: 3.4",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Topic :: Software Development :: Version Control"
"Programming Language :: Python :: 3.7",
"Topic :: Software Development :: Version Control :: Git"
]
)

Loading…
Cancel
Save