Code to download a different shapefile and make available

[srkobakian]

Can you provide the code to download other shapefiles, and convert to R objects, as part of this package? The SA2, SA4, .... all change periodically, and we will need to update the maps. If we are working on some historical data, we would need to have older shapefiles converted to R object. It would be really helpful to have this code available as part of the package.

[mdsumner]

That will be managed in ozmaps.data for the time being. Eventually I will use the rOpenSci package bowerbird to automate this for existing data and future data updates.

Keen for any feedback!

[srkobakian]

That would be great, I'm currently using SA2s for 2011, but will later migrate to using 2016.

I will keep playing and see if I can provide you more feedback, great work so far, I'm glad you've returned to developing it.

[mdsumner]

I was thinking about the installed data sets, I've only left the "NAME" on them - I meant to redo it so all the original attributes are there, for use with sf, if you happen to have any feeback.

Here, see how each file has its own non-generic naming scheme?

(This is using my local copy of the raw "piggyback" data from ozmaps.data, it's not in the package per se. )

x <- sf::read_sf("../ozmaps.data/extdata/abs-shp/CED_2018_AUST.shp")
names(x)

[1] "CED_CODE18" "CED_NAME18" "AREASQKM18" "geometry"

Those names annoy me, I feel like "CODE" and "NAME" is sensible, we can know what year/layer we've chosen by other means (it's classic de-normalization, ad hockery).

(Also AREASQKM18 is redundant because it's easily calculated, and it's potentially inaccurate because it won't update if we change the geometry, though that may be a valid reason for storing it ...).

Do you have any strong ideas about that? Here I would keep "CODE" and "NAME", and always use that column name, The other layers have different sets of names, e.g.

x <- sf::read_sf("../ozmaps.data/extdata/abs-shp/ILOC_2016_AUST.shp")
names(x) 

[1] "ILO_CODE16" "ILO_NAME16" "IAR_CODE16" "IAR_NAME16" "IRE_CODE16" "IRE_NAME16" >"STE_CODE16" [8] "STE_NAME16" "AREASQKM16" "geometry"

Final caveat: the official guidance is to use the Geopackage format (not shapefile or MapInfo or other), so these things might be better done there - I'll get to that soonish.

[srkobakian]

Depends on the use, you mention "we can know what year/layer we've chosen by other means" which is true, and I see this being helpful for regions that don't fall into hierarchies.

However, for my use, we need to know the SA2 name and code, as well as the upper hierarchy for each region. We use the SA3 and SA4 mainly for analysis, but also for plots, eg. as colours and facets.

So, personally I don't mind the code and name indicating the area, and even the year, if trying to plot transitions between the changing areas. They're helpful for my analysis when I'm using upper hierarchies and more than one census.

AREASQKM18 I haven't really used, so no strong feelings.

[mdsumner]

good points, thanks!

It seems like we need an "abs.data" package, and then I could wrap that and make the naming generic here. Would it be enough for the documentation to list the specific names, and point to the sources using those? These are necessarily not the same as the source data, being simplified etc.

[srkobakian]

I think having access to simplified shape files is great, especially when I'm jumping between just to have a look and don't want to simplify them all myself.
An abs.data package would be extremely helpful, there are a few of us in my department at Monash that would appreciate it. And would also be happy to be involved when you need it.

[mdsumner]

I just saw https://github.com/danwwilson/censaus which would be better for this package to leverage

[danwwilson]

@mdsumner I was just having a look at the ozmap_data() outputs and think it would be super helpful to have the codes in there. I stuck with the ABS led naming in censaus so that it was more transparent to users of other ABS ASGS data sets and would likely result in less digging through documents. I absolutely agree it makes for long winded names, but it reduces the friction to get up and running.

[mdsumner]

I sort of feel that that would be the "absmaps" package, though or censaus - and I'm happy to help get something like that going. My vague vision is that ozmaps.data is a good place to keep the code to get all this loot, and (say) store it in a piggyback artefact that these other packages can access when they build. The other thing is that I totally intend to leverage bowerbird for getting all these data sets, from that basis it would be quite a bit easier to spin off the right packages. You could just have a big old list of data, user-selects which ones and bowerbird goes and gets them so there's no package size overhead. That would be a good pairing of bowerbird and piggyback too.

I'm totally sympathetic to needing the native names, and another option is to have the data via a relation, you just run the "abs names" version of the data function to get it (though it's just via a join).

I also just don't have any time to spend on this unfortunately. I do really want ozmaps on CRAN, it fills a gap that I constantly hit - but the full-on ABS stuff is a bit much to me.