Skip to content

Commit 21d4bc9

Browse files
committed
Add documentation
1 parent b45d047 commit 21d4bc9

File tree

1 file changed

+33
-0
lines changed

1 file changed

+33
-0
lines changed

Documents/2016_Data_load.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# 2016 HMDA DATA LOAD
2+
3+
The 2016 LAR data is necessary for some macro edits that will make comparisons between the file submitted and the previous year's data.
4+
5+
In order to load the data, the 2017 panel should be loaded already. The LAR Loader will iterate through a file directory where the `HMDA` files reside,
6+
and send their contents to the `HMDA Platform`. The `HMDA` files must be in the pipe delimited format, with `.txt` extension and contain both Transmittal Sheet
7+
and Loan Application Register data, as per the [2017 Filing Instruction Guide](https://www.consumerfinance.gov/data-research/hmda/static/for-filers/2017/2017-HMDA-FIG.pdf)
8+
9+
10+
## Loading LAR data
11+
12+
By default, the loader will use `localhost` as the host where the HMDA Platform cluster is running. For remote systems, use the `HMDA_CLUSTER_HOST` environment variable.
13+
14+
When starting the cluster, take note of the port that Akka uses to communicate (i.e. 2551). This value needs to be set in an environment variable.
15+
The persistence cluster role needs to be active, and its dependencies (`Zookeeper`, `Cassandra`) running and properly configured.
16+
17+
```shell
18+
$ export HMDA_CLUSTER_PORT=2551
19+
$ sbt
20+
> project loader
21+
> run /tmp/sample_lar.txt 2016
22+
```
23+
24+
This will connect to the running cluster and send the contents of each file in the directory to the appropriate Actor responsible for storing `LARs` in the 2016 period.
25+
26+
## Troubleshooting
27+
28+
* In cases where there is no connection, make sure that the `HmdaSupervisor` and `HmdaQuerySupervisor` are receiving messages from the cluster client. Both of these actors
29+
need to have the Cluster Receptionist enabled. For more information please consult the current Akka documentation on [Cluster Client](https://doc.akka.io/docs/akka/current/scala/cluster-client.html)
30+
31+
* When loading large number of files, it might be necessary to increase the `hmda.persistent-actor-timeout` in the `Persistence-model` project's configuration file (or by passing the relevant
32+
runtime flag in a deployed application).
33+

0 commit comments

Comments
 (0)