An exercise in simplification
I recently needed to solve a problem:
Problem
There is a file which contains pairs of user ids and timestamps. Each line contains one pair. user ids are sorted in ascending order, and for each user id, timestamps are sorted in ascending order. Here is an example file:
For instance, in above example, user1 has three sessions: consisting of first three visits, next two visits and the next visit.
Output for above file would be:
Initial thought is to
Problem
There is a file which contains pairs of user ids and timestamps. Each line contains one pair. user ids are sorted in ascending order, and for each user id, timestamps are sorted in ascending order. Here is an example file:
user1 2012-09-18-14:30:29 user1 2012-09-18-14:30:55 user1 2012-09-18-14:31:12 user1 2012-09-18-15:31:12 user1 2012-09-18-15:31:12 user1 2012-09-18-16:31:12 user2 2012-09-19-14:30:29 user2 2012-09-19-14:30:55 user2 2012-09-19-14:31:12 user2 2012-09-19-15:31:12 user2 2012-09-19-15:31:13 user2 2012-09-19-15:31:16For each user, we need to output various session lengths. A session is defined as a sequence of consecutive visits so that no two visits are more than 30 minutes apart. We need not output the user ids.
For instance, in above example, user1 has three sessions: consisting of first three visits, next two visits and the next visit.
Output for above file would be:
3, 2, 1 3, 3First cut
Initial thought is to
- Scan through the file checking when the user id changes
- For a given user id, keep checking if next timestamp is more than 30 minutes after the previous one.
from datetime import datetime, timedelta
NEW = 'new'
def are_close(t1, t2):
fmt = '%Y-%m-%d-%H:%M:%S'
diff = datetime.strptime(t2, fmt) - datetime.strptime(t1, fmt)
return diff < timedelta(minutes = 30)
def write_details(sessions):
print ', '.join([str(len(session)) for session in sessions])
def write_one_id_details(viz_id, page_views):
if viz_id == NEW:
return
if len(page_views) == 0:
return
sessions = []
session = [page_views[0]]
if len(page_views) == 1:
sessions.append(session)
write_details(session)
return
for i in range(1, len(page_views)):
if are_close(page_views[i - 1], page_views[i]):
session.append(page_views[i])
else:
sessions.append(session)
session = [page_views[i]]
sessions.append(session)
write_details(sessions)
def process():
f = open('visits.txt')
curr_viz_id = NEW
page_views = []
for line in f:
viz_id, time = line.strip().split('\t')
if viz_id != curr_viz_id:
write_one_id_details(curr_viz_id, page_views)
curr_viz_id = viz_id
page_views = []
page_views.append(time)
write_one_id_details(curr_viz_id, page_views)
f.close()
if __name__ == '__main__':
process()
While this works, there are several problems with this code:- We have a variable called NEW to boostrap the process
- In write_one_id_details(), we need to initialize sessions to [page_views[0]]. This is hindrance in achieving harmony.
- When a loop ranges from 1 to n - 1, I do not like it. Loops should range from 0< to n - 1.
- sessions = [] or page_views = [] are indications of procedural programming. I would rather want easy to understand declarative style
