hgdb.py

Tue, 11 Aug 2015 19:12:30 +0300

author
Teemu Piippo <tsapii@utu.fi>
date
Tue, 11 Aug 2015 19:12:30 +0300
changeset 151
e24793fae424
parent 146
c17b82b1f573
child 154
df862cca1773
permissions
-rw-r--r--

Yet more fixes

146
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
1 '''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
2 Copyright 2015 Teemu Piippo
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
3 All rights reserved.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
4
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
5 Redistribution and use in source and binary forms, with or without
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
6 modification, are permitted provided that the following conditions
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
7 are met:
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
8
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
9 1. Redistributions of source code must retain the above copyright
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
10 notice, this list of conditions and the following disclaimer.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
11 2. Redistributions in binary form must reproduce the above copyright
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
12 notice, this list of conditions and the following disclaimer in the
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
13 documentation and/or other materials provided with the distribution.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
14 3. The name of the author may not be used to endorse or promote products
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
15 derived from this software without specific prior written permission.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
16
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
17 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
18 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
19 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
20 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
21 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
22 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
23 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
24 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
25 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
26 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
27 '''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
28
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
29 import os
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
30 import sqlite3
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
31 import hgpoll
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
32 from datetime import datetime
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
33 from configfile import Config
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
34
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
35 class HgCommitsDatabase (object):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
36 def __init__(self):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
37 dbname = Config.get_node('hg').get_value('commits_db', 'commits.db')
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
38 needNew = not os.path.isfile (dbname)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
39 self.db = sqlite3.connect (dbname)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
40
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
41 if needNew:
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
42 self.create_new()
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
43
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
44 def create_new (self):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
45 self.db.executescript ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
46 DROP TABLE IF EXISTS COMMITS;
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
47 DROP TABLE IF EXISTS REPOS;
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
48 DROP TABLE IF EXISTS REPOCOMMITS;
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
49 CREATE TABLE IF NOT EXISTS COMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
50 (
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
51 Node text NOT NULL,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
52 Dateversion text NOT NULL,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
53 PRIMARY KEY (Node)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
54 );
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
55 CREATE TABLE IF NOT EXISTS REPOS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
56 (
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
57 Name text NOT NULL,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
58 PRIMARY KEY (Name)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
59 );
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
60 CREATE TABLE IF NOT EXISTS REPOCOMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
61 (
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
62 Reponame text,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
63 Node text,
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
64 FOREIGN KEY (Reponame) REFERENCES REPOS(Name),
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
65 FOREIGN KEY (Node) REFERENCES COMMITS(Node)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
66 );
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
67 ''')
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
68
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
69 print ('Building commits.db...')
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
70 for repo in hgpoll.Repositories:
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
71 print ('Adding commits from %s...' % repo.name)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
72
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
73 for line in repo.hg ('log', '--template', '{node} {date|hgdate}\n').splitlines():
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
74 changeset, timestamp, tz = line.split(' ')
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
75 self.add_commit (repo, changeset, int (timestamp))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
76
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
77 self.commit()
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
78
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
79 def add_commit (self, repo, changeset, timestamp):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
80 dateversion = datetime.utcfromtimestamp (timestamp).strftime ('%y%m%d-%H%M')
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
81 self.db.execute ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
82 INSERT OR IGNORE INTO REPOS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
83 VALUES (?)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
84 ''', (repo.name,))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
85
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
86 self.db.execute ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
87 INSERT OR IGNORE INTO COMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
88 VALUES (?, ?)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
89 ''', (changeset, dateversion))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
90
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
91 self.db.execute ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
92 INSERT INTO REPOCOMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
93 VALUES (?, ?)
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
94 ''', (repo.name, changeset))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
95
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
96 def get_commit_repos (self, node):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
97 cursor = self.db.execute ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
98 SELECT Reponame
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
99 FROM REPOCOMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
100 WHERE Node LIKE ?
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
101 ''', (node + '%',))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
102
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
103 names = cursor.fetchall()
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
104 names = set (zip (*names)[0]) if names else set()
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
105 return [hgpoll.RepositoriesByName[name] for name in names]
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
106
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
107 def find_commit_by_dateversion (self, dateversion):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
108 cursor = self.db.execute ('''
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
109 SELECT Node
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
110 FROM COMMITS
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
111 WHERE Dateversion = ?
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
112 ''', (dateversion,))
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
113 result = cursor.fetchone()
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
114 return result[0] if result else None
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
115
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
116 def commit(self):
c17b82b1f573 Mercurial handling major overhaul. Also get some stuff ready for Python 3
Teemu Piippo <tsapii@utu.fi>
parents:
diff changeset
117 self.db.commit()

mercurial