# Implicit Nonlinear Dynamics : Autoregulation

The following is another example on how to use the ImplicitOptimizer describing a biological autoregulation process using two coupled implicit equations.

using DataDrivenDiffEq
using ModelingToolkit
using OrdinaryDiffEq
using LinearAlgebra
using Plots
gr()

@parameters begin
t
α = 1.0
β = 1.3
γ = 2.0
δ = 0.5
end

@variables begin
x[1:2](t) = [20.0; 12.0]
end

x = collect(x)
D = Differential(t)

eqs = [
D(x) ~ α/(1+x)-β*x;
D(x) ~ γ/(1+x)-δ*x;
]

sys = ODESystem(eqs, t, x, [α, β, γ, δ], name = :Autoregulation)

x0 = [x => 20.0; x => 12.0]

tspan = (0.0, 5.0)

de_problem = ODEProblem(sys, x0, tspan)
de_solution = solve(de_problem, Tsit5(), saveat = 0.005);
plot(de_solution)

As always, we start by defining a DataDrivenProblem and a sufficient basis for sparse regression.

dd_prob = ContinuousDataDrivenProblem(de_solution)

eqs = [
polynomial_basis(x, 4); D.(x); x .* D(x); x .* D(x)
]

basis = Basis(eqs, x, independent_variable = t, implicits = D.(x))

plot(dd_prob)

Next to varying over different sparsity penalties, we want also to batch our data using the DataSampler. We define a train-test split of 80-20 for our data and batch the resulting training data into 10 minibatches, allowing shuffled and repeated values. Our goal is to find the model with the minimal error.

sampler = DataSampler(
Split(ratio = 0.8), Batcher(n = 10, shuffle = true, repeated = true, batchsize_min = 30)
)

res = solve(dd_prob, basis, ImplicitOptimizer(STLSQ(1e-1:1e-1:9e-1)), by = :min, sampler = sampler, digits = 1)
Linear Solution with 2 equations and 10 parameters.
Returncode: solved
L₂ Norm error : [2.9071727536337378e-12, 6.5944082037062e-29]
AIC : [-26570.403909412264, -64933.63439441865]
R² : [0.9999999999999999, 1.0]
Model ##Basis#507 with 2 equations
States : x(t) x(t)
Parameters : 10
Independent variable: t
Equations
Differential(t)(x(t)) = (p₁ + p₂*x(t) + p₃*x(t)*x(t)) / (-p₄ - p₅*x(t))
Differential(t)(x(t)) = (p₆ + p₇*x(t) + p₈*x(t)*x(t)) / (-p₉ - p₁₀*x(t))

We have recovered the correct equations of motion! Another visual check using the problem and the result yields

system = result(res)
@named ode = ODESystem(equations(system), t, x, parameters(system));
ode_prob = ODEProblem(ode, x0, tspan, parameter_map(res));

prediction = solve(ode_prob, Tsit5(), saveat = 0.2);
plot(de_solution, label = ["Groundtruth" nothing])
scatter!(prediction, label = ["Prediction" nothing])

## Copy-Pasteable Code

using DataDrivenDiffEq
using ModelingToolkit
using OrdinaryDiffEq
using LinearAlgebra

@parameters begin
t
α = 1.0
β = 1.3
γ = 2.0
δ = 0.5
end

@variables begin
x[1:2](t) = [20.0; 12.0]
end

x = collect(x)
D = Differential(t)

eqs = [
D(x) ~ α/(1+x)-β*x;
D(x) ~ γ/(1+x)-δ*x;
]

sys = ODESystem(eqs, t, x, [α, β, γ, δ], name = :Autoregulation)

x0 = [x => 20.0; x => 12.0]

tspan = (0.0, 5.0)

de_problem = ODEProblem(sys, x0, tspan)
de_solution = solve(de_problem, Tsit5(), saveat = 0.005);

eqs = [
polynomial_basis(x, 4); D.(x); x .* D(x); x .* D(x)
]

basis = Basis(eqs, x, independent_variable = t, implicits = D.(x))

sampler = DataSampler(
Split(ratio = 0.8), Batcher(n = 10, shuffle = true, repeated = true, batchsize_min = 30)
)

res = solve(dd_prob, basis, ImplicitOptimizer(STLSQ(1e-1:1e-1:9e-1)), by = :min, sampler = sampler, digits = 1)
print(res) #hide
print(result(res)) #hide

system = result(res)
@named ode = ODESystem(equations(system), t, x, parameters(system));
ode_prob = ODEProblem(ode, x0, tspan, parameter_map(res));

prediction = solve(ode_prob, Tsit5(), saveat = 0.2);

# This file was generated using Literate.jl, https://github.com/fredrikekre/Literate.jl