Plugin initialization and global caching

[gudaoxuri]

Requires a plugin that:

1. Synchronize redis data to memory when the plugin is initialized

function _M.check_schema(conf)
   local check_ok, check_err = core.schema.check(schema, conf)
   if not check_ok then
       core.log.error("Configuration parameter error")
       return false, check_err
   end
   -- Repeat call
   local _, redis_err = m_redis.init(conf.redis_host, conf.redis_port, conf.redis_database, conf.redis_timeout, conf.redis_password, nil, nil)
   if redis_err then
       core.log.error("Connect redis error", redis_err)
       return false, redis_err
   end
   -- Repeat call
   m_init.init(conf.cache_resources, conf.cache_change_resources, conf.cache_change_resources_timer_sec)
   return true 
 end

1. Synchronize redis change data to memory at periodic intervals

function _M.init(cache_resources, cache_change_resources, cache_change_resources_timer_sec)
  m_redis.hscan(cache_resources, "*", 100, function(k, v)
      local res = m_utils.split(k, "##")
      m_resource.add_res(res[2], res[1], json.decode(v))
  end)
  -- Repeat register
  ngx.timer.every(cache_change_resources_timer_sec, function()
      core.log.debug("Fetch changed resources")
      m_redis.scan(cache_change_resources, 10, function(_, v)
          local res_value, err = m_redis.hget(cache_resources, v)

1. Compare uri and memory data after receiving a request

local core = require("apisix.core")

-- This cache object will exist for each coroutine!
local RESOURCES = {}

function _M.add_res(res_action, res_uri, auth_info)
  res_action = string.lower(res_action)
  core.log.info("Add resource [" .. res_action .. "][" .. res_uri .. "]")
  local items = parse_uri(res_uri)

Currently these requirements can be implemented, but there are the following problems:

1. plugin initialization will start several coroutines, how to avoid repeated execution of synchronization
2. how to implement a global cache to avoid multiple copies in memory

Thanks.

[gudaoxuri]

Requires a plugin that:

1. Synchronize redis data to memory when the plugin is initialized

function _M.check_schema(conf)
   local check_ok, check_err = core.schema.check(schema, conf)
   if not check_ok then
       core.log.error("Configuration parameter error")
       return false, check_err
   end
   -- Repeat call
   local _, redis_err = m_redis.init(conf.redis_host, conf.redis_port, conf.redis_database, conf.redis_timeout, conf.redis_password, nil, nil)
   if redis_err then
       core.log.error("Connect redis error", redis_err)
       return false, redis_err
   end
   -- Repeat call
   m_init.init(conf.cache_resources, conf.cache_change_resources, conf.cache_change_resources_timer_sec)
   return true 
 end

1. Synchronize redis change data to memory at periodic intervals

function _M.init(cache_resources, cache_change_resources, cache_change_resources_timer_sec)
  m_redis.hscan(cache_resources, "*", 100, function(k, v)
      local res = m_utils.split(k, "##")
      m_resource.add_res(res[2], res[1], json.decode(v))
  end)
  -- Repeat register
  ngx.timer.every(cache_change_resources_timer_sec, function()
      core.log.debug("Fetch changed resources")
      m_redis.scan(cache_change_resources, 10, function(_, v)
          local res_value, err = m_redis.hget(cache_resources, v)

1. Compare uri and memory data after receiving a request

local core = require("apisix.core")

-- This cache object will exist for each coroutine!
local RESOURCES = {}

function _M.add_res(res_action, res_uri, auth_info)
  res_action = string.lower(res_action)
  core.log.info("Add resource [" .. res_action .. "][" .. res_uri .. "]")
  local items = parse_uri(res_uri)

Currently these requirements can be implemented, but there are the following problems:

1. plugin initialization will start several coroutines, how to avoid repeated execution of synchronization
2. how to implement a global cache to avoid multiple copies in memory

Thanks.

The initialization of worker can be skipped with process.type() ~= "worker", but the value type of the cache data share ngx.shared is too simple to satisfy the requirement.

Should we use wasm instead?

[tzssangglass]

1. plugin initialization will start several coroutines, how to avoid repeated execution of synchronization

process.type() == "privileged agent"

or

ngx.worker.id() == 0

[gudaoxuri]

This problem has been solved.

[tzssangglass]

2. how to implement a global cache to avoid multiple copies in memory

This problem is somewhat complicated, and ngx.shared is currently the most direct and reliable way to do it. wasm does not solve this problem as I recall.

[gudaoxuri]

I want to share large json objects in multiple coroutines, there is no good solution.

I would like to use wasm, but rust currently does not support redis operations in wasm environment.

[tzssangglass]

I want to share large json objects in multiple coroutines, there is no good solution.

I know, I think we should avoid sharing large json and serialization and deserialization between multiple workers.

[gudaoxuri]

I want to share large json objects in multiple coroutines, there is no good solution.

I know, I think we should avoid sharing large json and serialization and deserialization between multiple workers.

I think it is necessary to share objects between multiple workers:

1. The redis connection pool can be initialized by the "privileged agent" when the gateway is started and shared by multiple workers. Now, like the limit implementation, each worker holds a pool of connections(If I understand it correctly).

apisix/apisix/plugins/limit-count/limit-count-redis.lua

Lines 54 to 65 in ab4fe88

	function _M.incoming(self, key)
	local conf = self.conf
	local red = redis_new()
	local timeout = conf.redis_timeout or 1000 -- 1sec
	core.log.info("ttl key: ", key, " timeout: ", timeout)
	red:set_timeouts(timeout, timeout, timeout)
	local ok, err = red:connect(conf.redis_host, conf.redis_port or 6379)
	if not ok then
	return false, err
	end

2. Based on the abac model of local cache data support, each worker constructs a cached copy which is very wasteful of space.

[tzssangglass]

I think it is necessary to share objects between multiple workers:

Sharing data between multiple workers is a known problem in Nginx/OpenResty, and there is no perfect solution. I agree with you about the optimization, but there is no better implementation.