- andrewclegg January 9, 2019 at 5:47 pm
Would it be possible to add a flag in the .shr files that indicates that an “Uncal/IF overload” condition existed for a given sweep?
I perform offline processing of data and could use the flag to ignore any sweeps that were acquired during overload. Also, without the flag, when playing back previously recorded data in Spike, there is no way to indicate that uncal/overload conditions existed when a sweep was originally acquired.
Can this be a new feature in a future version of Spike? Or is there some way to accomplish this already?
As always, thanks!
AndrewModeratorAndrew January 10, 2019 at 11:27 am
Thank you for the request. You are correct in noting that the overload condition is not stored in the sweep file. I can look into adding this in Spike.
It sounds like you are saving the files in Spike and parsing them in your own application? If so, simply storing the overload condition would be adequate for your needs? If there was a setting to not save overloaded sweeps, would that also be adequate?
I look forward to your response.
andrewclegg January 10, 2019 at 4:27 pm
- This reply was modified 5 days, 20 hours ago by andrewclegg.
Thanks for the quick response.
Generally, yes, I acquire and save in Spike, and process the data in my own Python applications. I sometimes also look at the recorded data in Spike. Probably 75%/25% split. In either case, it would be good to know which sweeps are potentially corrupted (or uncalibrated) due to overload.
Preferentially, all sweeps would be kept, and those that are overloaded would be flagged. This would allow continuous data acquisition, even if some of the sweeps should be considered suspect, and then I can deal with them accordingly and as desired in post-processing.
The next best solution would be to simply not record sweeps that are overloaded, but personally I would prefer the first solution, since to me it’s always better to collect as much data as possible and deal with imperfections accordingly in post-processing. For some applications, I could imagine it’s better to have a sweep even if it’s uncal than to be missing data completely (for example, when trying to capture the exact time and general nature of an infrequent short burst, for example).
Of course, the truly ideal solution is to have both: the option to either record and flag overloaded sweeps, or not record them.
I believe there’s 16 bytes of unused data in each sweep (unint64_t reserved). Could the overload condition be encoded in part of those unused bits?
By the way, when decimation is used, how are overloaded sweeps that occur during the decimation period handled? Is there any special treatment or are they handled like any other sweep? If the latter, it would probably be good to flag the affected decimated sweeps too, if they are possibly corrupted.
I always try to avoid acquiring important data in overload conditions, but sometimes I’m on the edge due to dynamic range considerations, and it would usually be good to be able to identify uncal data if it happens.
AndrewModeratorAndrew January 14, 2019 at 8:59 am
Thank you for the information Andy. I have made notes of your requests and will see about getting this functionality in a future release of Spike.
Regardsandrewclegg January 14, 2019 at 9:39 am
Perfect, thanks. BTW, I really appreciate all the work you put into Spike. It’s a great piece of software, and keeps getting even better with each release.
You must be logged in to reply to this topic.