I'm the author of Sidekiq and not trying to debate; I'm simply ignorant of how Erlang, BEAM and the ecosystem solve these common problems, which usually persist no matter which language you use.
%% ping a remote node to connect to it
pong = net_adm:ping('foo@127.0.0.1'),
%% start my application on the remote node
_ = spawn('foo@127.0.0.1', application, start, [my_app]),
%% my_app is a gen_server, an otp process that abstracts
%% async request/response and makes it look more like
%% synchronous function calls
%% this example assumes the gen_server accepts the JobArgs
%% and immediately returns 'ok' as acknowledgement, but does
%% the work asynchronously
ok = gen_server:call({my_app, 'foo@127.0.0.1'}, {do_some_work, JobArgs})
this example assumes you're uninterested in the responses, but if you wanted to receive the results you could either send a pid (process identifier) along with the request that would be setup to collect the results and do any further processing or you could spawn a local process that made the actual call and blocked until it received the results
as for scheduling a job a week in the future, the naive way would be to setup a timer that, when it expires, spawns a process that can execute an arbitrary function. no one would do this, normally, however. you'd probably want to use something like sidekiq that can store the job off node and poll regularly to see if there's any jobs you should run
It doesn't have a scheduled job API yet, but there is a note in the readme that explains how to do this with quantum-elixir: https://github.com/joakimk/toniq#how-do-i-run-scheduled-or-r....
Start the processes that execute the code, get notified when they are finished. Literally a couple of lines of Erlang/Elixir. Not some large framework that does all that for you, it's in the language. It is built to run a lot of tasks in parallel and across many machines.
> How do you schedule a job to run in a week?
Like anyone else, put it in some permanent storage like redis. You can also keep it in memory, you can also use mnesia which is a distributed database that comes with Erlang.
my_list_of_job
|> distribute_tasks()
|> Enum.map(fn x -> Task.Supervisor.async(x, my_list_of_job) end)
|> Enum.map(fn x -> Task.await(x, your_timeout) end)
There is a long running project to work on that to produce a nice job queue lib on Elixir Slack but noone on the interested group have time for now.
To schedule a job to run in a week you have a couple way. An easy first approach is to use Process.send_after/3 to send yourself a message after a week and handle that message in a week. Of course if you want persistence, etc, you will have to do it a bit differently, but not so much....
edit : now use the fn form instead of the shorthand & for ease of comprehension for non seasoned Elixir devs
I heard that Erlang/OTP (and thus Elixir) can transparently distribute work no matter where it gets physically executed but I'd just love to see an example -- freely accessible code, demo site, everything.