Skip to content

Tc issue 80 #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 33 commits into
base: develop
Choose a base branch
from
Open

Tc issue 80 #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b1100bf
Adding Kalman predictions for frequency based services. Moved alot of…
scrudden Aug 18, 2018
ff5fbee
First running version of freq using kalman for travel times.
scrudden Aug 25, 2018
3ad19fc
Added code to predict dwell times for frequency based services using …
scrudden Sep 2, 2018
411574a
reset polling rate.
scrudden Sep 2, 2018
0cc8b11
Broader check to see if vehicle has passed first stop.
scrudden Sep 8, 2018
8712482
Merge branch 'develop' into tc_issue_80
scrudden Sep 8, 2018
1e88aff
Merge branch 'develop' of https://github.com/TheTransitClock/transiti…
scrudden Sep 9, 2018
46d7103
Fix compile is with merge.
scrudden Sep 9, 2018
92d864f
Fix issue #107
scrudden Sep 9, 2018
3d76d9d
Merge branch 'tc_issue_107' into tc_issue_80
scrudden Sep 9, 2018
ef1fe85
Merge branch 'develop' into tc_issue_80
scrudden Sep 9, 2018
5d1fd8a
Differentiate trip by freqStartTime.
scrudden Sep 12, 2018
ea050e2
Tighten up filter values and discard dwell model if return silly values.
scrudden Sep 13, 2018
a5d0783
Add square view for loop routes
Sep 14, 2018
779ed8f
Add patternType to ApiShape
Sep 14, 2018
3a4fbf0
Out of schedule if schAdh>120 seg
Sep 14, 2018
dd8cd38
Remove model when producing silly values.
scrudden Sep 18, 2018
90dc055
Merge remote-tracking branch 'vperez/tc_issue_110' into tc_issue_80
scrudden Sep 18, 2018
5888b11
Correction of direction if undefined
Sep 18, 2018
be8f6d3
Merge remote-tracking branch 'vperez/tc_issue_110' into tc_issue_80
scrudden Sep 18, 2018
2cff99b
refactor package names for prediction methods.
scrudden Sep 23, 2018
e3af82b
Add start time for vehicle with isScheduledService=false
Sep 25, 2018
8f69090
Merge remote-tracking branch 'vperez/tc_issue_110' into tc_issue_80
scrudden Sep 26, 2018
89abe12
Correction of variable data to vehicle
Sep 28, 2018
2607076
Check that frestart times are equal when matching arrival to departur…
scrudden Oct 1, 2018
6ac36a3
Merge remote-tracking branch 'vperez/tc_issue_110' into tc_issue_80
scrudden Oct 2, 2018
2f8abff
Set prediction type to TheTransitClock.
scrudden Oct 2, 2018
8aa1750
Fix for #113
scrudden Oct 11, 2018
6d610e1
Add agency_id to primary key for agency.
scrudden Jan 12, 2019
6373d58
Add database create SQL scripts. These really should be deleted as th…
scrudden Jan 12, 2019
34b4cd8
Revert "Add database create SQL scripts. These really should be delet…
scrudden Jan 12, 2019
7efdbf3
Add correct SQL files with latest database create scripts.
scrudden Jan 12, 2019
f3147a4
Merge pull request #119 from TheTransitClock/tc_issue_117
scrudden Jan 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 80 additions & 52 deletions transitclock/src/main/java/org/transitclock/applications/Core.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
import org.transitclock.core.dataCache.TripDataHistoryCacheFactory;
import org.transitclock.core.dataCache.VehicleDataCache;
import org.transitclock.core.dataCache.ehcache.StopArrivalDepartureCache;
import org.transitclock.core.dataCache.ehcache.TripDataHistoryCache;
import org.transitclock.core.dataCache.ehcache.scheduled.TripDataHistoryCache;
import org.transitclock.core.dataCache.frequency.FrequencyBasedHistoricalAverageCache;
import org.transitclock.core.dataCache.scheduled.ScheduleBasedHistoricalAverageCache;
import org.transitclock.db.hibernate.DataDbLogger;
Expand Down Expand Up @@ -409,6 +409,77 @@ public static void startRmiServers(String agencyId) {
PredictionAnalysisServer.start(agencyId);
HoldingTimeServer.start(agencyId);
}

static private void populateCaches() throws Exception
{
Session session = HibernateUtils.getSession();

Date endDate=Calendar.getInstance().getTime();

/* populate one day at a time to avoid memory issue */
for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);
if(StopArrivalDepartureCacheFactory.getInstance()!=null)
{
logger.debug("Populating StopArrivalDepartureCache cache for period {} to {}",startDate,endDate);
StopArrivalDepartureCacheFactory.getInstance().populateCacheFromDb(session, startDate, endDate);
}

endDate=startDate;
}
endDate=Calendar.getInstance().getTime();


if(cacheReloadStartTimeStr.getValue().length()>0&&cacheReloadEndTimeStr.getValue().length()>0)
{
if(TripDataHistoryCacheFactory.getInstance()!=null)
{
logger.debug("Populating TripDataHistoryCache cache for period {} to {}",cacheReloadStartTimeStr.getValue(),cacheReloadEndTimeStr.getValue());
TripDataHistoryCacheFactory.getInstance().populateCacheFromDb(session, new Date(Time.parse(cacheReloadStartTimeStr.getValue()).getTime()), new Date(Time.parse(cacheReloadEndTimeStr.getValue()).getTime()));
}

if(FrequencyBasedHistoricalAverageCache.getInstance()!=null)
{
logger.debug("Populating FrequencyBasedHistoricalAverageCache cache for period {} to {}",cacheReloadStartTimeStr.getValue(),cacheReloadEndTimeStr.getValue());
FrequencyBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, new Date(Time.parse(cacheReloadStartTimeStr.getValue()).getTime()), new Date(Time.parse(cacheReloadEndTimeStr.getValue()).getTime()));
}
}else
{
for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);

if(TripDataHistoryCacheFactory.getInstance()!=null)
{
logger.debug("Populating TripDataHistoryCache cache for period {} to {}",startDate,endDate);
TripDataHistoryCacheFactory.getInstance().populateCacheFromDb(session, startDate, endDate);
}

if(FrequencyBasedHistoricalAverageCache.getInstance()!=null)
{
logger.debug("Populating FrequencyBasedHistoricalAverageCache cache for period {} to {}",startDate,endDate);
FrequencyBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, startDate, endDate);
}

endDate=startDate;
}
}
endDate=Calendar.getInstance().getTime();

for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);

if(ScheduleBasedHistoricalAverageCache.getInstance()!=null)
{
logger.debug("Populating ScheduleBasedHistoricalAverageCache cache for period {} to {}",startDate,endDate);
ScheduleBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, startDate, endDate);
}

endDate=startDate;
}
}

/**
* The main program that runs the entire Transitime application.!
Expand Down Expand Up @@ -436,57 +507,14 @@ public static void main(String[] args) {

Date endDate=Calendar.getInstance().getTime();

/* populate one day at a time to avoid memory issue */
for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);

logger.debug("Populating StopArrivalDepartureCache cache for period {} to {}",startDate,endDate);
StopArrivalDepartureCacheFactory.getInstance().populateCacheFromDb(session, startDate, endDate);

endDate=startDate;
}
endDate=Calendar.getInstance().getTime();





if(cacheReloadStartTimeStr.getValue().length()>0&&cacheReloadEndTimeStr.getValue().length()>0)
{
logger.debug("Populating TripDataHistoryCache cache for period {} to {}",cacheReloadStartTimeStr.getValue(),cacheReloadEndTimeStr.getValue());
TripDataHistoryCacheFactory.getInstance().populateCacheFromDb(session, new Date(Time.parse(cacheReloadStartTimeStr.getValue()).getTime()), new Date(Time.parse(cacheReloadEndTimeStr.getValue()).getTime()));

logger.debug("Populating FrequencyBasedHistoricalAverageCache cache for period {} to {}",cacheReloadStartTimeStr.getValue(),cacheReloadEndTimeStr.getValue());
FrequencyBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, new Date(Time.parse(cacheReloadStartTimeStr.getValue()).getTime()), new Date(Time.parse(cacheReloadEndTimeStr.getValue()).getTime()));
}else
{
for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);

logger.debug("Populating TripDataHistoryCache cache for period {} to {}",startDate,endDate);
TripDataHistoryCacheFactory.getInstance().populateCacheFromDb(session, startDate, endDate);

logger.debug("Populating FrequencyBasedHistoricalAverageCache cache for period {} to {}",startDate,endDate);
FrequencyBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, startDate, endDate);

endDate=startDate;
}
}
// TODO just here to check values reasonable.
logger.info(FrequencyBasedHistoricalAverageCache.getInstance().toString());

endDate=Calendar.getInstance().getTime();

for(int i=0;i<CoreConfig.getDaysPopulateHistoricalCache();i++)
{
Date startDate=DateUtils.addDays(endDate, -1);

logger.debug("Populating ScheduleBasedHistoricalAverageCache cache for period {} to {}",startDate,endDate);
ScheduleBasedHistoricalAverageCache.getInstance().populateCacheFromDb(session, startDate, endDate);

endDate=startDate;
// populate caches to be used by prediction methods.
try {
populateCaches();
if(TripDataHistoryCacheFactory.getInstance()!=null)
TripDataHistoryCacheFactory.getInstance().logCache(logger);

} catch (Exception e) {
logger.error("Failed to populate cache.", e);
}

// Initialize the core now
Expand Down
11 changes: 6 additions & 5 deletions transitclock/src/main/java/org/transitclock/core/ArrivalDepartureGeneratorDefaultImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,10 @@
import org.transitclock.core.dataCache.TripDataHistoryCacheFactory;
import org.transitclock.core.dataCache.VehicleStateManager;
import org.transitclock.core.dataCache.ehcache.StopArrivalDepartureCache;
import org.transitclock.core.dataCache.ehcache.TripDataHistoryCache;
import org.transitclock.core.dataCache.ehcache.scheduled.TripDataHistoryCache;
import org.transitclock.core.dataCache.frequency.FrequencyBasedHistoricalAverageCache;
import org.transitclock.core.dataCache.scheduled.ScheduleBasedHistoricalAverageCache;
import org.transitclock.core.holdingmethod.HoldingTimeGeneratorDefaultImpl;
import org.transitclock.core.holdingmethod.HoldingTimeGeneratorFactory;

import org.transitclock.core.predAccuracy.PredictionAccuracyModule;
Expand Down Expand Up @@ -326,10 +327,7 @@ protected Arrival createArrivalTime(VehicleState vehicleState,
block,
tripIndex,
stopPathIndex, freqStartDate);
if(vehicleState.getVehicleId().equals("969"))
{
System.out.println("hello");
}

updateCache(vehicleState, arrival);
logger.debug("Creating arrival: {}", arrival);

Expand Down Expand Up @@ -377,6 +375,9 @@ private void updateCache(VehicleState vehicleState, ArrivalDeparture arrivalDepa
HoldingTimeGeneratorFactory.getInstance().handleDeparture(vehicleState, arrivalDeparture);

}
if(HoldingTimeGeneratorDefaultImpl.getOrderedListOfVehicles("66")!=null)
logger.info("ORDER:"+HoldingTimeGeneratorDefaultImpl.getOrderedListOfVehicles("66").toString());

/*
if(HoldingTimeGeneratorFactory.getInstance()!=null)
{
Expand Down
13 changes: 12 additions & 1 deletion transitclock/src/main/java/org/transitclock/core/Indices.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import org.transitclock.db.structs.Trip;
import org.transitclock.db.structs.TripPattern;
import org.transitclock.db.structs.VectorWithHeading;
import org.transitclock.gtfs.DbConfig;

/**
* This private class is for keeping track of the trip, path, and segment
Expand Down Expand Up @@ -97,7 +98,17 @@ public Indices(SpatialMatch spatialMatch) {

public Indices(ArrivalDeparture event) {

this.block=event.getBlock();

Block block=null;
if(event.getBlock()==null)
{
DbConfig dbConfig = Core.getInstance().getDbConfig();
block=dbConfig.getBlock(event.getServiceId(), event.getBlockId());
}else
{
block=event.getBlock();
}
this.block=block;
this.tripIndex = event.getTripIndex();
this.stopPathIndex = event.getStopPathIndex();

Expand Down
12 changes: 7 additions & 5 deletions transitclock/src/main/java/org/transitclock/core/PredictionGenerator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
import org.transitclock.core.dataCache.TripDataHistoryCacheInterface;
import org.transitclock.core.dataCache.TripKey;
import org.transitclock.core.dataCache.ehcache.StopArrivalDepartureCache;
import org.transitclock.core.dataCache.ehcache.TripDataHistoryCache;
import org.transitclock.core.dataCache.ehcache.scheduled.TripDataHistoryCache;
import org.transitclock.db.structs.ArrivalDeparture;
import org.transitclock.db.structs.AvlReport;
import org.transitclock.db.structs.Block;
Expand Down Expand Up @@ -283,10 +283,12 @@ protected List<TravelTimeDetails> lastDaysTimes(TripDataHistoryCacheInterface ca
if (arrival != null && departure != null) {

TravelTimeDetails travelTimeDetails=new TravelTimeDetails(departure, arrival);

times.add(travelTimeDetails);

num_found++;

if(travelTimeDetails.getTravelTime()!=-1)
{
times.add(travelTimeDetails);
num_found++;
}
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions transitclock/src/main/java/org/transitclock/core/TravelTimeDetails.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ public class TravelTimeDetails {
private static final IntegerConfigValue maxTravelTime =
new IntegerConfigValue(
"transitclock.core.maxTravelTime",
10 * Time.MS_PER_MIN,
30 * Time.MS_PER_MIN,
"This is a maximum allowed for travel between two stops. Used as a sanity check for cache and predictions.");

private static final Logger logger = LoggerFactory
Expand Down Expand Up @@ -53,7 +53,7 @@ public boolean sanityCheck()
{
long travelTime=this.arrival.getTime()-this.getDeparture().getTime();

if(travelTime<0||travelTime>maxTravelTime.getValue())
if(travelTime<=0||travelTime>maxTravelTime.getValue())
{
return false;
}else
Expand Down
Loading