Airflow DAG Scheduled date is a week behind

There's a chapter on Scheduling in the Airflow documentation, which states:

Note that if you run a DAG on a schedule_interval of one day, the run stamped 2016-01-01 will be trigger soon after 2016-01-01T23:59. In other words, the job instance is started once the period it covers has ended.

Let’s Repeat That The scheduler runs your job one schedule_interval AFTER the start date, at the END of the period.

You are experiencing exactly this: today (2019-05-06) a DagRun is created for the latest "completed" interval, meaning the week starting on 2019-04-29.

Thinking about it like this might help: if you want to process some data periodically, you need to start processing it after the data is ready for that period.


Airflow schedule a dag at the ending of each interval with execution time as the starting of that interval. So usually execution_time=schedule_time-interval.

For example, in your dag, the last interval was 2019-04-29T14:00:00 to 2019-05-06T14:00:00 and its execution only get scheduled on 2019-05-06T14:00:00 with execution time as 2019-04-29T14:00:00. It is the usual working of airflow. It's not sure how your dag did run with 2019-04-29T14:00:00 before MAY 6th 2 PM, as you mentioned in your question. Maybe you changed the dag interval or made a manual trigger.

Tags:

Python

Airflow