Skip to content

Job queue time units #452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Nov 8, 2016
Merged

Conversation

voider1
Copy link
Contributor

@voider1 voider1 commented Nov 2, 2016

This enables users to add intervals in seconds (default), minutes and hours for their Jobs.
It also enables users to define on which days the Job should be run (all by default).
It's backwards compatible!

Copy link

@evieluvsrainbows evieluvsrainbows left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, Wesley. Great job! :)

@voider1
Copy link
Contributor Author

voider1 commented Nov 3, 2016

Thank you, @KamranMackey!

jsmnbom
jsmnbom previously requested changes Nov 3, 2016
Copy link
Member

@jsmnbom jsmnbom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work on this PR!
However, there are some things that would have to be changed before we can merge it.
I really like the idea of being able to specify which days the jobs should run (and several users have actually requested it), but this implementation is simply not up to the standards of the rest of the library.

from threading import Thread, Lock, Event
from queue import PriorityQueue, Empty


class Days(Enum):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weeks unfortunately don't always start on a monday. In some locales it's sundays.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bomjacob Does this actually matter all that much? I know this might come off as rude, but I don't see why this matters. It's a really tiny thing.

Copy link

@evieluvsrainbows evieluvsrainbows Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And as far as I know, it would require too much code to be able to do an change to where Sunday is day 0 in certain locales, as we don't even know how many locales have Sunday as the first day of the week.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I'm not sure, but I'd say that the entirety of North America is a pretty big demographic...
Question is if it even matters... as long as the code below correctly knows that a monday is a monday (and not necesarily a 0) then everything is fine... I'm not entirely sure that python is that smart though :/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bomjacob I've already done my research on this, Python2 and Python 3 both return the .weekday() from 0-6 where monday is 0 and sunday is 6.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case across all locales (that is, python doesn't automatically change that weekday depending on where it's run) then it shouldn't matter and this should be fine...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the official Datetime docs that is the case, so I think we're good on the mapping of the days.

@@ -56,6 +68,12 @@ def __init__(self, bot, prevent_autostart=None):
""":type: float"""
self._running = False

def dt_time_to_seconds(self, dt_time):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use .total_seconds() here?

@@ -126,7 +156,9 @@ def tick(self):
self.logger.debug('Running job %s', job.name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This debug would wrongly say that it's running a job even though it's not, if it's on an excluded day.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've fixed this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using a separate get_days function here, because if I put it in the class, it won't be filtered out by the list comprehension, and I won't be able to easily get the values from the attributes defined in the class.

@@ -169,6 +178,26 @@ def test_inUpdater(self):
finally:
u.stop()

def test_time_units(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split this into multiple test that only test one thing each.
E.g. have one that tests if seconds (remember to test floats too btw) and another test which tests if datetime objects work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot find a reliable way to test the float, one time it succeeds, one time it fails. Do you know a way in which I can reliably test floats? I know for certain that floats work, is not testing them an option, because if the test starts on the wrong millisecond and rounds the wrong way, the test will fail.

@@ -21,10 +21,22 @@
import logging
import time
import warnings
import datetime
from enum import Enum
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enums are a new feature that appeared in 3.4. Note that this library also supports python 2.7.
While they have been backported and are available on pypi, I would rather not add a whole dependency for something that is used so very little.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bomjacob I don't think one extra dependency will hurt people. And Enums are not used so "very little". And there's no way to track how often Enums are being used either. But I know that they're used more than your "so very little" stat.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Granted, we use enums elsewhere in the project... but a enum in python could just as well be a class with some attributes.... There's a reason it wasn't added until 3.4... it's pretty much unnecessary. And of cause if there was a library that would make our library easier to maintain, easier to read or easier to develop on, we wouldn't hesitate to add it as a dependency - but Enums simply aren't that.

Copy link
Contributor Author

@voider1 voider1 Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bomjacob I'd personally like to keep the enum library, it's supported and a very clean way of doing it, it isn't a very big dependency, and could be used more throughout the project as it grows.

An alternative would look like:

>>> def enum(**named_values):
...     return type('Enum', (), named_values)
...
>>> Days = enum(mon=0, tue=1, wed=2, thu=3, fri=4, sat=5, sun=6)
>>> Days.mon
0
>>> class Days(object):
...     def __init__(self, day):
...         if day not in (Days.mon , Days.tue, Days.wed, Days.thu, Days.fri, Days.sat, Days.sun):
...             raise ValueError('gender not valid')
...         self.day = day
...

I don't like this pattern because it relies upon the integers too much and has way more overhead.
In my original example the Enum doesn't relie on the integers too much, only to check if one of the values of the Days which were supplied with the Job is today. The Enum subclass returns a special Enumeration object instead of the literal integer value. This is a lot nicer for error handling because people should use the supplied Days Enum subclass. If you don't do this, people could just pass in integers from 0-6 and it'd accept. As far as I know, we can't check if a value comes from a certain class if we do this with a plain new-style class. With an Enum we can check if it comes from the class it should come from and enforce people to use the Days Enum subclass.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you want to prevent users from using integers? It's not really very python like to require inputs to be of a special type if you don't have to. My recommendation would be as follows:
Make the Days a simple class with the weekdays as class variables (attributes), this is what we use for enums elsewhere in the project and it's worked fine (if you believe that it absolutely wouldn't work here, please do correct me).
Just raise a ValueError if 0 < day > 7 inside ``Job's init` or `JobQueue`'s `put`.

In the future we might upgrade to use Enums everywhere, but for now it's simply not necessary.

Copy link
Contributor Author

@voider1 voider1 Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code is more often read than written. Preventing users from using plain integers makes it clearer what someone's trying to accomplish when looking over the code.

For example:

import datetime

test_job = Job(callback, datetime.time(hour=3), repeat=True, days=(0, 4, 6))

If I make my bot open source and someone reads this, it's very unclear what the original developer tries to do without first reading the documentation of this repo. It's also very confusing if you have different locals (like you pointed out) and you assume that 0 is sunday instead of monday. Of course, we would like to make the developer use our Days class. But he/she isn't enforced to do it, and I think we should assume people wouldn't if they don't really have to.

This code:

import datetime

from telegram.ext.jobqueue import Days

test_job = Job(callback, datetime.time(hour=3), repeat=True, days=(Days.mon, Days.fri, Days.sun))

This code is a little longer, but it's entirely self-documenting! A developer who is unknown with the codebase will instantly know how it works. The syntax is clear, the functionality is clear, and we're enforcing them to use the Enum from the project instead of integers like we saw in the first example. I think this is more Pythonic, because "there should be one-- and preferably only one --obvious way to do it", except for string formatting.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree to use enum because Python 2 is going to drop its support soon. Python 3 is the new standard.

Copy link
Member

@jsmnbom jsmnbom Nov 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see @tsnoam comment below. Bottom line is that for now, don't use Enums unless they're part of future (which I don't think they are).

self.callback = callback
self.interval = interval
self.repeat = repeat
self.context = context

if not isinstance(days, tuple):
err_msg = "The 'days' argument should be of type 'tuple'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please just write the msg directly in the exception, it simply looks a bit cleaner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everywhere or only this instance?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everywhere preferably :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the max line length for the project? I use 84 as my personal standard, so when a message is too long, I use a variable.

Copy link

@evieluvsrainbows evieluvsrainbows Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@voider1 Max line length is 99 characters.

@voider1
Copy link
Contributor Author

voider1 commented Nov 3, 2016

Thank you! I will address all the changes.

Copy link
Contributor Author

@voider1 voider1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last commit message should be: "Adding error handling for the method 'timedelta_to_seconds'."

@jsmnbom
Copy link
Member

jsmnbom commented Nov 6, 2016

Here's some comments from @tsnoam from our dev team group:
all from phone so I don't want to comment in github so I write here:

  1. import numbers # or something like that

isinstance(x, number)

  1. raise ValueError : don't use \ at eol
  2. days=tuple(Days)
    no need: day for day in Days
  3. race condition around midnight with line 156
  4. was too lazy to read the unitests
  5. all in all looks good. the only thing is that they use enum and unless it's part of "future" we already require as a dependency it can be implemented without enum which is a trivial change to the code.

@voider1
Copy link
Contributor Author

voider1 commented Nov 6, 2016

@bomjacob

Point 2 doesn't work if it isn't a subclass of Enum. If the bottom-line is not using Enum, I'd have to manually get the ints from the class or implement an iterator in the class.

>>> class Days(object):
...     mon = 0
...     tue = 1
...
...
>>> tuple(Days)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    tuple(Days)
TypeError: 'type' object is not iterable

Point 3, do you mean that when I set the time for Sunday at 00:00 it'll never execute because then it's Monday? If that's the case, place an heads up in the documentation when this gets added that 00:00 is a day later, and that they should execute at 23:59 instead of 00:00 or just change the day.

Point 5 those are a lot of small changes, sadly, not entirely relying on integers, making sure users use the Days Enum. You guys should really think about making Enums part of the project. They're a lot nicer to work with then plain new-style classes, and do not add overhead in the code with implementing iterators or manually getting al the values from the class (which I'll have to do now). I'll make them new-style classes for now. But please, really just consider adding this to the whole project. It's a small dependency which makes these things a lot easier.

@voider1
Copy link
Contributor Author

voider1 commented Nov 6, 2016

@bomjacob I've addressed all the issues you've mentioned. Could you review it again, please?

Copy link
Member

@jh0ker jh0ker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the way this is going. I requested some changes, but most of them are small.

One more thing: We had multiple requests to add the possibility of adding Jobs that run at a certain time instead of defining next_t and interval as time deltas (not referring to the timedelta class here).
While I'm not asking you to implement that, I would appreciate if you could give it some thought, and, if required, modify this PR so that it doesn't make the implementation of that any harder than necessary.

sun = 6


def get_days():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what this is supposed to do, but it's still unreadable. I think the best solution would be to do something like we did in MessageEntity. Other ideas?

from threading import Thread, Lock, Event
from queue import PriorityQueue, Empty


class Days(object):
mon = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we switched to a class with constants, the correct naming convention would be UPPER_CASE if I'm not mistaken.

@@ -56,6 +72,12 @@ def __init__(self, bot, prevent_autostart=None):
""":type: float"""
self._running = False

def timedelta_to_seconds(self, tdelta):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method seems to be redundant, especially since this looks like a purely internal method and you already check isinstance(var, datetime.timedelta)

return tdelta.total_seconds()
except AttributeError:
raise AttributeError("timedelta_to_seconds only takes a datetime.timedelta object")

def put(self, job, next_t=None):
"""Queue a new job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the docstring so it mentions that next_t can also be of type timedelta

self.callback = callback
self.interval = interval
self.repeat = repeat
self.context = context

print(days)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug print


"""
job_queue = None

def __init__(self, callback, interval, repeat=True, context=None):
def __init__(self, callback, interval, repeat=True, context=None, days=get_days()):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my earlier comment, this would change to days=Days.ALL_DAYS (or maybe even days=Days.EVERY_DAY? Your choice ^^)

@@ -208,22 +242,36 @@ class Job(object):
callback (function): The callback function that should be executed by the Job. It should
take two parameters ``bot`` and ``job``, where ``job`` is the ``Job`` instance. It
can be used to terminate the job or modify its interval.
interval (float): The interval in which this job should execute its callback function in
seconds.
interval (float or datetime.time): The interval in which this job should execute its
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you switched to timedelta instead of time, please update this docstring to reflect that change.

try:
job.run(self.bot)
for day in job.days:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone would, for whatever reason, put a day twice (like Job(..., days=(Days.mon, Days.mon))) this would run the job twice on those days. I'm not sure this is a bug or a feature, but it could be easily changed by changing the type of Job.days from tuple to set. Your choice.

@voider1
Copy link
Contributor Author

voider1 commented Nov 8, 2016

@jh0ker I adressed all the changes and fixed them.

@jh0ker jh0ker dismissed jsmnbom’s stale review November 8, 2016 22:30

Changes have been adressed

@voider1
Copy link
Contributor Author

voider1 commented Nov 8, 2016

@jh0ker About the idea of running them at a certain time, I'll maybe do that after this PR gets merged.

@jh0ker
Copy link
Member

jh0ker commented Nov 8, 2016

LGTM :)

@jh0ker jh0ker merged commit 68e87db into python-telegram-bot:master Nov 8, 2016
@voider1 voider1 deleted the job-queue-time-units branch November 8, 2016 22:40
@voider1 voider1 restored the job-queue-time-units branch November 8, 2016 22:40
@voider1 voider1 deleted the job-queue-time-units branch November 9, 2016 00:12
@github-actions github-actions bot locked and limited conversation to collaborators Aug 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy