SOLR Admin Tasks
Do some command line steps:
# First create the directories for the new core
sudo mkdir -p /data/solr/data/biocache-reindex-2/data
# Copy the previous solrconfig, schema there
sudo cp -a /data/solr/data/biocache/conf/ /data/solr/data/biocache-reindex-2/conf
# Set the right perms
sudo chown -R solr:solr /data/solr/data/biocache-reindex-2/
Later you can add the core via the SOLR UI interface:
Now we'll do a copy of the index on the filesystem.
Remove previous reindex:
rm -rf /data/solr/solr-create/biocache/data0*
rm -rf /data/solr/solr-create/biocache/data1*
rm -rf /data/solr/merged_*
FIXME: Should biocache-config.properties point to the new process during the local reindex? like following the previous created core:
solr.home=http://index.gbif.es:8983/solr/biocache-reindex-2
FIXME: Any other additional manual step with schemas, etc?
No we can start the reindex:
#
# biocache index-local-node options:
#
# -t threads
# -ms, mergesegments, The number of output segments. No merge output is produced when 0.
# -wc, writercount, The number of index writers.
# -wt, writerthreads, The number of threads for each indexing writer. There is 1 writer for each -t.
# -wb, writerbuffer, Size of indexing write buffer. The default is + writerBufferSize, {
# -pt, processthreads, The number of threads for each indexing process. There is 1 process for each -t.
# -pb, processbuffer, Size of the indexing process buffer.
# -r, writerram, Ram allocation for each writer
# -ws, writersegmentsize, Maximum number of occurrences in a writer segment. There is 1 writer for each -t.
# -ps, pagesize, The page size for the records.
# -max, maxrecords, Maximum number of records to index. This is mainly for testing new indexing.
# basic text index
biocache index-local-node -t 4 -max 1000
# With more options, sample from: https://github.com/AtlasOfLivingAustralia/biocache-store/issues/329
biocache index-local-node -t 8 -pt 8 -wc 2 -wt 2 -r 1024 -ps 500 -pb 500 -wb 500 -ws 100000000 -max 50000
# or a Full reindex
biocache index-local-node -t 4 -max -1
The previous generated index need to manually copied into a remote SOLR (in this example).
rsync --delete -aH --rsync-path "sudo rsync" /data/solr/merged_0/ 172.16.16.200:/data/solr/data/biocache-reindex-2/data/index/
ssh 172.16.16.200 "sudo chown -R solr:solr /data/solr/data/biocache-reindex-2/data/index/"
When your data is ready (for instance after a BIE or with the previous biocache-cli local indexing), you can swap the cores to put the core in production:
