>If you have a signal + noise then estimate an ARMA model based on
>this, you will also model the additive noise.

Yes, exactly. Try some of the Total Least Squares ARMA techniques.
Greg

Reply by Rune Allnor●September 10, 20082008-09-10

On 10 Sep, 09:50, "Nick32" <fatmansincl...@hotmail.com> wrote:

> Hi
>
> Thanks for all of your responses.
>
> The data I am trying to fit an (8,5) ARMA model to is a physiological
> signal.
>
> The reason for specifically using an (8,5) ARMA model comes from a
> physiological modelling theory.

Never expect a model to fit data just because you have an analytical
model. Real-life measurements *never* comply to analytical models.

> Essentially, we want to fit (8,5) ARMA models to segments of data and see
> how the model system properties (poles and zeros) change over time. By
> relating these results back to the theory we hope to gain some
> physiological insight.

Famous last words. As you already have discovered, the properties of
the data depend at least as much on noise and imperfections as on
whatever parameters you look for.
Not to say that what you want is impossible, but it will take a lot
more skill and effort than you might have abticipated, and the
results might not be quite as relevant as you expect.

> As the amplitude of our signal is very small (microvolts), the signal to
> noise ratio is low (10-20dB).

Have a look at the sensors. I don't have much hands-on experience
with hardware design, but I did a student project ages ago where
we converted a voltage source (a piezoelectric element which was
nothing more than a weird capacitor, really) to a current source
by attaching a high-impedance OpAmp as a voltage follower.
As a general rule of thumb do whatever is in your grasp,
technically and economically, to get good data. While DSP
can help you out from time to time, the easiest problems
to deal with are still those that never occur.

> A lot of the testing I have been doing has been with synthetic data.
> Basically, I have been generating data by filtering Gaussian white noise
> with �known (8,5) ARMA models to create a clean data set and then adding
> noise to it to create a noisy data set. I then fitted models (using the
> methods mentioned previously) to the clean and noisy data and compared
> them.

That's the only way to start. Try and fit higher-order ARMA
models (say, ARMA(10,8)) to the low-SNR data and see if the
'best' 5 poles and 3 zeros behave better.
Rune

Reply by Nick32●September 10, 20082008-09-10

Hi
Thanks for all of your responses.
The data I am trying to fit an (8,5) ARMA model to is a physiological
signal.
The reason for specifically using an (8,5) ARMA model comes from a
physiological modelling theory.
Essentially, we want to fit (8,5) ARMA models to segments of data and see
how the model system properties (poles and zeros) change over time. By
relating these results back to the theory we hope to gain some
physiological insight.
As the amplitude of our signal is very small (microvolts), the signal to
noise ratio is low (10-20dB).
A lot of the testing I have been doing has been with synthetic data.
Basically, I have been generating data by filtering Gaussian white noise
with known (8,5) ARMA models to create a clean data set and then adding
noise to it to create a noisy data set. I then fitted models (using the
methods mentioned previously) to the clean and noisy data and compared
them.
Thanks again.
Nick

Reply by Rune Allnor●September 9, 20082008-09-09

On 9 Sep, 15:15, "Nick32" <fatmansincl...@hotmail.com> wrote:

> Hi
>
> I am trying to fit an (8,5) ARMA model to a time series in the presence of
> additive white noise. The SNR is around 10-20dB.

What do you try to achieve? Why ARMA? Why ARMA(8,5)?

> So far I have tried the armax function from the Matlab system
> identification toolbox, the ARMASA toolbox for Matlab and a reduced
> statistics algorithm. Unfortunately I have found that these methods are
> unreliable when the SNR is less than about 60dB, and the models produced
> contain significant errors.

Well, I've never been impressed by matlab's DSP stuff...

> I have also tried some methods based on the modified Yule Walker
> equations, which avoid the zero lag of the autocorrelation, but these too
> are unreliable below 60dB and can return unstable models.
>
> What I would like to know (and hopefully someone here can help me) is:
>
> -Is it actually possible to get an accurate ARMA model when the SNR is
> around 10-20dB?

I've worked with AR models (not ARMA) with SNR in the range 5-10 dB
without awful lots of difficulties. Well, I had hundreds of data sets
so I could see trends across the whole data set instead of relying
on individual results. Which certainly helped.
It's not impossible to get results at those SNRs, but it's not
straight-forward.

> -Are there any "tried and true" robust methods for ARMA parameter
> estimation at low SNR?

No. You need to try and test lots of different combinations
of methods, and see which combos produce the desired results.
Which means you will need to do the tests in controlled
settings where you know the properties of the data and thus
know how well the methods performed. Which means you need
to work with simulated data.

> -Can anyone suggest a good resource for ARMA estimation from noisy
> observations, or a resource that lists possible ARMA estimation methods

Kay wrote several books in the '80s and '90s; Therrien's 1992
book is my personal favourite (but it has already been out of
print for more than a decade). But those deal *almost* exclusively
with AR methods. Therrien has a reasonable treatise on practical
ARMA methods, but don't expect miracles.

> -Alternatively, are there any reliable methods for estimating the
> amplitude �of the additive white noise component, so that it can
> compensated for.

Again, what are you trying to do? You are approaching
specialist's territory where every single detail can
be the decider of whether what you want can be done at
all, and if so, what approaches might work.
Rune

Reply by HardySpicer●September 9, 20082008-09-09

On Sep 10, 1:15 am, "Nick32" <fatmansincl...@hotmail.com> wrote:

> Hi
>
> I am trying to fit an (8,5) ARMA model to a time series in the presence of
> additive white noise. The SNR is around 10-20dB.
>
> So far I have tried the armax function from the Matlab system
> identification toolbox, the ARMASA toolbox for Matlab and a reduced
> statistics algorithm. Unfortunately I have found that these methods are
> unreliable when the SNR is less than about 60dB, and the models produced
> contain significant errors.
>
> I have also tried some methods based on the modified Yule Walker
> equations, which avoid the zero lag of the autocorrelation, but these too
> are unreliable below 60dB and can return unstable models.
>
> What I would like to know (and hopefully someone here can help me) is:
>
> -Is it actually possible to get an accurate ARMA model when the SNR is
> around 10-20dB?
>
> -Are there any "tried and true" robust methods for ARMA parameter
> estimation at low SNR?
>
> -Can anyone suggest a good resource for ARMA estimation from noisy
> observations, or a resource that lists possible ARMA estimation methods
>
> -Alternatively, are there any reliable methods for estimating the
> amplitude of the additive white noise component, so that it can
> compensated for.
>
> Thanks
> Nick

If you have a signal + noise then estimate an ARMA model based on
this, you will also model the additive noise.

Reply by Yihui●September 9, 20082008-09-09

ARMA is used for spectrum estimation. Like all learning process, ARMA
learns the high frequency first. If the signal is too noisy, ARMA only
learns the noise, thus become unstable.
I always smooth a noisy signal before apply ARMA.
By the way, Yule Walker method should guarantee the stability, I think.
Yihui

Reply by Vladimir Vassilevsky●September 9, 20082008-09-09

Nick32 wrote:

> Hi
>
> I am trying to fit an (8,5) ARMA model to a time series in the presence of
> additive white noise. The SNR is around 10-20dB.

To begin with, what are you trying to accomplish?

> So far I have tried the armax function from the Matlab system
> identification toolbox, the ARMASA toolbox for Matlab and a reduced
> statistics algorithm. Unfortunately I have found that these methods are
> unreliable when the SNR is less than about 60dB, and the models produced
> contain significant errors.

The analytical methods are the simplified tractable approaches to the
non-tractable original problem of maximum likelihood. Because of that,
in the presense of noise and distortion, the results of the methods can
be very confusing.

> I have also tried some methods based on the modified Yule Walker
> equations, which avoid the zero lag of the autocorrelation, but these too
> are unreliable below 60dB and can return unstable models.
>
> What I would like to know (and hopefully someone here can help me) is:
>
> -Is it actually possible to get an accurate ARMA model when the SNR is
> around 10-20dB?
>
> -Are there any "tried and true" robust methods for ARMA parameter
> estimation at low SNR?
>
> -Can anyone suggest a good resource for ARMA estimation from noisy
> observations, or a resource that lists possible ARMA estimation methods
>
> -Alternatively, are there any reliable methods for estimating the
> amplitude of the additive white noise component, so that it can
> compensated for.

The most robust method is the brute force optimization of the parameters
of the model (if this search is computationally feasible). The AR, ARMA,
FDLS or any other method can provide the starting point for the search.
From my experience, it is unlikely to get the correct results from the
one shot application of an analytical method.
Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

Reply by Nick32●September 9, 20082008-09-09

Hi
I am trying to fit an (8,5) ARMA model to a time series in the presence of
additive white noise. The SNR is around 10-20dB.
So far I have tried the armax function from the Matlab system
identification toolbox, the ARMASA toolbox for Matlab and a reduced
statistics algorithm. Unfortunately I have found that these methods are
unreliable when the SNR is less than about 60dB, and the models produced
contain significant errors.
I have also tried some methods based on the modified Yule Walker
equations, which avoid the zero lag of the autocorrelation, but these too
are unreliable below 60dB and can return unstable models.
What I would like to know (and hopefully someone here can help me) is:
-Is it actually possible to get an accurate ARMA model when the SNR is
around 10-20dB?
-Are there any "tried and true" robust methods for ARMA parameter
estimation at low SNR?
-Can anyone suggest a good resource for ARMA estimation from noisy
observations, or a resource that lists possible ARMA estimation methods
-Alternatively, are there any reliable methods for estimating the
amplitude of the additive white noise component, so that it can
compensated for.
Thanks
Nick