Greetings, is it possible to use SAN for time series forecasting?

[APCunha]

Greetings,

I was wondering if it is possible to adapt your SAN implementation for a regression problem, like time series forecasting. For example, the prediction of the price of something having X inputs and then do the feature ranking of those inputs according to their importance.
If it is indeed possible, can you shed some light on it?

Thanks in advance!

[Petkomat]

Just to confirm: your examples are $(x, y)$ where $x = (x_1, \dots, x_n)$ are the inputs, $y$ is the price, and $x_i$ and $y$ are time series?

[APCunha]

Yes, exactly

[SkBlaz]

The simplest way is via autoregression; simply represent each instance with values for past n time points - these are your features. Value at time t is the target. Of course, initial part of the data will be partially incomplete depending on the window size. Direct mapping - san(X,Y), most likely won't work as there is no lstm like mechanism (at least not explicit)

[APCunha]

First of all, apologies for the lateness in this response. I think I managed to do some simple adapting, since I'm a novice in ML, and finally got the model to spit out some attention results for my problem. Specifically, since I have 7 inputs (like hour, month, price the day before, price one week before, etc), with the get_mean_attention_weights function I got 7 attention results, as expected. My question now is that if it is possible to do some further adapting and give the model the ability to return attention values regarding, for example, a specific month, which means, get the attention for specific sets of the data, maybe by changing some aspect of the matrix in the forward_attention function?

[SkBlaz]

@APCunha sorry for the late reply, you could retrieve attention vectors after each time period - it would show incremental change in attention in time ..