Usage questions ..

[choward94002]

Greets all!  A quick question while I work with this ST4 wonder! I understand that I can schedule a project for multiple or single nights to control when it completes, as well as ongoing to keep repeating the session again and again.  There is also (apparently only for comets and minor planets) the ability to get multiple observations per night ... but what if I want to get repeating completions per night?  

In other works, say my project of a DSO can complete with session time of 60min, but the object will be in the appropriate IQ area for 130min that evening.  I have set the exposure to be "Ongoing" so what I would like to do would be to have ST4 schedule two runs of the session, just like it does with multiple observations, so I would finish the night with two project completions rather than one ... changing the object type to Astrometry or Photometry doesn't ungrey the Exposure Goals checks from which I would control this capability ... ideas?

Thx!

[theskyhound]

Hi,

There is a way to do what you want to do, but in practice it is seldom necessary, so I suspect there may of a misunderstanding lurking somewhere. Generally if you schedule a project it will schedule as much time for the project as there is available. Available means, during a period of time when the object is within its acceptable IQ time period and there are no other projects scheduled.

I suppose if a higher priority project was scheduled in the middle of the available time, you may wish to schedule before and after that other project. Is that what you are asking about?

Regardless, the auto scheduler isn't yet advanced enough to do this on its own, but you can schedule by hand. To schedule a project to start at a specific time, use the start/end time selections. You can either drag the vertical lines from the ends of the NightBar, or enter the start and/or end times manually. Then select your project and click the Schedule button. Assuming the time period you selected is within the acceptable IQ period, it should start scheduling at the start time you selected.

If there is a higher IQ period that starts after the start time your selected, it may prefer to start later, when the higher IQ period begins. You may want to just let it do that. But if you want to force it, set the end time to just prior to the start of the higher IQ period. Then schedule.

Once I get the first update finished I am going to turn my attention to making video tutorials which should help with this sort of thing.

[choward94002]

Hi,

There is a way to do what you want to do, but in practice it is seldom necessary, so I suspect there may of a misunderstanding lurking somewhere. Generally if you schedule a project it will schedule as much time for the project as there is available. Available means, during a period of time when the object is within its acceptable IQ time period and there are no other projects scheduled.

I suppose if a higher priority project was scheduled in the middle of the available time, you may wish to schedule before and after that other project. Is that what you are asking about?

Regardless, the auto scheduler isn't yet advanced enough to do this on its own, but you can schedule by hand. To schedule a project to start at a specific time, use the start/end time selections. You can either drag the vertical lines from the ends of the NightBar, or enter the start and/or end times manually. Then select your project and click the Schedule button. Assuming the time period you selected is within the acceptable IQ period, it should start scheduling at the start time you selected.

If there is a higher IQ period that starts after the start time your selected, it may prefer to start later, when the higher IQ period begins. You may want to just let it do that. But if you want to force it, set the end time to just prior to the start of the higher IQ period. Then schedule.

Once I get the first update finished I am going to turn my attention to making video tutorials which should help with this sort of thing.

Cool!  As an FYI, what I'm doing to accomplish this now is to duplicate the project 'A' for the object, so I have two projects A and A' ... both of them now have an IQ A window of 130min, so the scheduler will put A from 0:00 to 60:00 and then schedule A' from 61:00 to 121:00 ... nice!

[theskyhound]

Wow. That really should not be necessary! Something doesn't seem right. Why does the first one stop at 60? When you get the chance, please consider posting a screen capture of the Scheduler window so that I can understand what is happening. I suspect that you don;'t have something set up right. Its always better to do things correctly than to kludge them.

[choward94002]

Wow. That really should not be necessary! Something doesn't seem right. Why does the first one stop at 60? When you get the chance, please consider posting a screen capture of the Scheduler window so that I can understand what is happening. I suspect that you don;'t have something set up right. Its always better to do things correctly than to kludge them.

No no ... I *want* it to stop after 60min, at that point it's gotten what it needs to hit the target SNR and it's good, I have no need for it to linger longer.  Consider the minor planet photometry project, where I want to get a series of lights throughout the evening to see if something "moved".  I'm not interested in getting the super best picture of the minor planet, I just want to get an image good enough to be able to compare it to another image taken 10min later or so.  ST4 has just that capability (and the user is warned that iTelescope won't appreciate that kind of schedule with a gap in the middle, in fact) ...

Now consider that I have a DSO that I want to get multiple lights of over a period of several weeks.  Each individual light I want to get an SNR of 60 for, and on this night ST4 calculates that it will take 60min for that to happen.  Assume that this evening the DSO will be in an IQ A position for a total of 130min ... based on that I actually have the potential to take two of my 60min SNR 60 lights during that 130min interval which is what I want; two lights today, two more tomorrow, maybe ten more by the end of the week, hopefully I'll hit my goal of 50 SNR 60 lights of that object by the time the object is no longer in IQ A range for the season ...

I can't use the multiple exposure checkbox in the Exposure Goals section since the DSO isn't a minor planet and it's greyed out, so by duplicating the project I have ST4 schedule project "A" for the first part of the IQ A period, when the scheduler gets to project "A'" it says "oh look, there's a period available right near the end of the IQ A range for this, great!" and schedules project A' right after it schedules project A.  End result: I will get two SNR 60 lights of the DSO object ...

There's another advantage of duplicating projects: say that tonight my IQ 4 windows is only 95 minutes ... project A makes it find, but project B only has 35 min to image, not enough unless the project is set to "ongoing" which means that tomorrow with another IQ 4 window I'll have enough time to get another complete A light but ST4 will also be able to finish up the ongoing A' project, giving me a total of three DSO lights over two days ... woohoo!

Hope this makes sense!

[theskyhound]

Ok, I see what's going on now. But I hate kludges though. You should not have to do something as extreme as setting up two different projects for the same object. I think the issue is that you are setting the Imaging Project to complete at 60 SNR, when in fact, you apparently what a lot more than that. The SNR that you specify is supposed to be the total SNR that you want to achieve, not just the SNR on a single night.

What I am running into when I speak to people about ST4, now that they have had some time to start using it, is that they naturally want to force it to work the way that they have always worked. But ignoring a few shortcomings, the way ST4 works came about entirely from using the model to get the best possible images. So if the user forces another way of working on to it, they will eventually become unhappy. Not only will ST4 be ill suited to the way they want to work, but they won't get the optimum results. For many people, using ST4 is going to mean changing how the work.

There is no need to achieve your final SNR all at once. You could use a 5 minute exposure one night, another next week, three more next month, and more next year. So here is how it would suggest doing your observations within the ST4 framework:

1. Set your project to only complete after the final SNR that you want, say 100. As long as the quality of the images is similar, it does not matter if they were taken on different nights.

2. On the Basic tab of the project, set "Expose On" to multiple nights. This tells the Scheduler that we don;t have to get the whole 100 SNR in one night.

3. When you schedule, let the Scheduler do its thing. Don't force it to do what you want. It will adapt sub exposures to the conditions, and choose the number of images on a nightly basis in order to achieve the final result that you want. It will squeeze the maximum SNR out of the time you have each night. Your job is to set the parameters for it will work within, such as the minimum IQ. Use the Priority to give it hints about what to do first. Under some circumstances you may want to force the sub exposure times to always be the same, so that they will stack better. But only if you have good reason to.

4. Log every observation. The hardest adjustment for some people is to make the investment in logging their observations as they go. If you take an extra few minutes to log them, you get a long-term benefit in return. By logging, the Scheduler will know how many more observations are needed to reach your target SNR. Eventually it will tell you that the project is completed. After that you will always have an archive of your observations.

Remember the SkyTools 4 official motto: "Projects are meant to be completed, never deleted."

Please let me know how things go and how you adapt as you learn the software. I know that it some ways ST4 will need to adapt too.

[choward94002]

Ok, I see what's going on now. But I hate kludges though. You should not have to do something as extreme as setting up two different projects for the same object. I think the issue is that you are setting the Imaging Project to complete at 60 SNR, when in fact, you apparently what a lot more than that. The SNR that you specify is supposed to be the total SNR that you want to achieve, not just the SNR on a single night.

Fair enough, and I hate kludges too ...

What I am running into when I speak to people about ST4, now that they have had some time to start using it, is that they naturally want to force it to work the way that they have always worked. But ignoring a few shortcomings, the way ST4 works came about entirely from using the model to get the best possible images. So if the user forces another way of working on to it, they will eventually become unhappy. Not only will ST4 be ill suited to the way they want to work, but they won't get the optimum results. For many people, using ST4 is going to mean changing how the work.

ST4 does indeed change the way things are done, and so far I'm more than happy ... I think I'm going to get at least 20% more lights per evening than before (once the full moon clears, of course) ..

< good advice >

Please let me know how things go and how you adapt as you learn the software. I know that it some ways ST4 will need to adapt too.

Sure!  Maybe you can give me some pointers at how to adjust my current workflow ... so here's what I do today (the "old way") ...

For each object (project) I'm shooting I'm going to be stacking 50+ lights (likely 100+ over time as I get better) with a goal of an SNR of 60.  I want to be able to go back and reprocess the lights as I get better and if I want to emphasize detail, or color, or whatever ... so each evening I for each project I'll take a number of lights on one side of the meridian or the other as well as the flats to go with those lights.  I track the average temperature (from my focuser thermocouple) and the SQM for that evening, and in the morning stack those lights plus darks from a master library for that temperature plus the latest bias frames into a master flat, master dark and master bias for that evening.  I then take those exposures and sort them by SQM (graduated in .02 units) to be later processed and stacked.

Using ST4 today, what I've done is for each DSO I'm interested in I've created twelve projects, each referring to a unique location. Those locations are East and West of meridian (using the obstructed horizon feature) and then with SQM values going through the range I've found at the location (from 21.32 to 21.24 by .02, 6 total).  Each project is defined by SNR, has a goal SNR of 80, is multi-night using "fine" detail (different for project types, but basically "fine" detail) and has a seeing value of 1.5" using "A" IQ and auto subexposure.  I group like-SNR and like-meridian side projects into Target Lists (so, East SQM 21.32, West SQM 21.26, etc.)

My site SNR's don't vary much night to night but do vary month to month, so based on the last evening's worse recorded SQM (typically around 2300, when the high clouds reflect light from a light dome from a nearby city) so I'll load up the appropriate Target List for the appropriate meridian side and generate two schedules.  ST4 is very good about picking what's good, so my scopes remain busy but in the morning some of the projects for objects imaged with SQM X will have made some progress and I'll have some lights ...

Some of the items from my current workflow that I need to work around ("kludge") are ...

- Single meridian side for the evening; that's to prevent the mirror flop that plagues 14's and to a lesser extent 11's
- Varying SQM's from night to night ... postprocessing is much easier if the background noise is as similar as possible, and I've had really bad results with some lights where the background SQM's vary.  It's easier for me to simply take a bunch of lights at a known SQM than try to convince PixInsight to sort things out

Hope this help and maybe you've got some ideas I can try!  Thx!

[theskyhound]

Hi,

Thank you for explaining your workflow in detail. It helps me better understand what needs to be done. I am working on a feature to avoid meridian flips, but I haven't quite yet figured out how it should work. If you don't mind, I'l like to ask some questions.

1. When you spoke about two different "target lists" and scheduling them, that didn't make sense to me. Did you really mean two different Observing Programs? I ask this because you can't really schedule from a target list, but people are clever, so you never know. ;-)

2. You spoke of "fine" detail, as if it is a setting somewhere in SkyTools. Someone else (or maybe you) mentioned the same thing on another thread. What do you mean by "fine" detail?

3. Your approach to scheduling primarily via SQM readings is quite unusual. Am I reading right that your sky brightness varies from 21.2 to 21.3 (more or less). If so, that puts your location at a fairly dark site. I understand that noise from the sky may dominate anyways, but sky noise is a big part of the SNR model that SkyTools uses. So I am not sure that it is justified to put sky brightness, particularly over such a small range, as the top consideration. I'd be interested in hearing more. Are you directly concerned about sky brightness itself, or are you using it for a proxy for something else, such as transparency?

4. Regarding meridian flips. I think I understand but I want to ask to be sure. Is your goal that all images be taken on the same side of the pier for a given project? Or is it such a pain to do a meridian flip because of refocusing or whatever, that you want to do as few as possible on a given night?

5. I have the beginnings of an idea in mind to schedule in such a way as to limit the number of meridian flips. It works on the assumption that your goal is to minimize the number of flips during the night. So based on that assumption, how many times do you normally flip during the night? How do you decide when to do the flip?

Thanks!

[choward94002]

Hi,

Thank you for explaining your workflow in detail. It helps me better understand what needs to be done. I am working on a feature to avoid meridian flips, but I haven't quite yet figured out how it should work. If you don't mind, I'l like to ask some questions.

1. When you spoke about two different "target lists" and scheduling them, that didn't make sense to me. Did you really mean two different Observing Programs? I ask this because you can't really schedule from a target list, but people are clever, so you never know. ;-)

On the main dialog there are several tabs, "Target Selection", "Scheduler", "Real Time Imaging" and "ACP Expert" ... on the "Target Selection" tab there is a combo where you can pick a grouping of targets to choose from; NGC, Messier, Skyhound, RASC Best, etc. which simply groupings of DSO's.  On the target selection dialog a user also has the ability to create their own "target lists" which is what I've done ... so, on the "Target Selection" tab when I select the "target list" to view I select one from SiteA, SiteB, etc. based on the seeing criteria (see below)

2. You spoke of "fine" detail, as if it is a setting somewhere in SkyTools. Someone else (or maybe you) mentioned the same thing on another thread. What do you mean by "fine" detail?

In the "Image Project" dialog, there are several tabs, "Basic Properties", "Composition and Tiles", "Exposure Goals and Filters" and "Observations" ... on the "Exposure and Goals" tab there is an "Expose For" group box, inside of which you can choose the level of detail for the object; for a galaxy that would be "Nucleus", "Main extent/ arms", "Halo", "Faint Halo" which correspond to a level of detail; "Nucleus" would be very coarse, "Main extent/arm" is better (but the nucleus might be blown out), "Halo" is finer (but the nucleus is definitely blown out and the arms are likely getting bad) and "Faint Halo" is highest level but everything else is blown out ... nebulae have a similar grouping of "level of detail", I simply use the "fine" with for galaxies would correspond to "Main extent/ arms"

3. Your approach to scheduling primarily via SQM readings is quite unusual. Am I reading right that your sky brightness varies from 21.2 to 21.3 (more or less). If so, that puts your location at a fairly dark site. I understand that noise from the sky may dominate anyways, but sky noise is a big part of the SNR model that SkyTools uses. So I am not sure that it is justified to put sky brightness, particularly over such a small range, as the top consideration. I'd be interested in hearing more. Are you directly concerned about sky brightness itself, or are you using it for a proxy for something else, such as transparency?

You're correct both that I'm able to image from a dark site where I have my scope "farm" running semi-remotely as well as the SQM varying by around .15 based on the time of year, mainly influenced by high clouds brought in by the monsoon season in early summer there.  I agree that sky noise is taken into account by ST4, but remember there's a bigger ecosystem here than just exposure calculations of lights ... namely, remember that I need to be able to stack these together at some point, and stacking has it's own rules ...

Recall the way that stacking programs work: very simplistically given a set of lights, they essentially do an averaging of each registered pixel across all of the supplied lights, adding pixels as needed and removing them as needed but with the assumption of a certain degree of uniformity between the lights.  One of the big things to watch out for with stacking is to not "poison" the stack by giving it an outlier light that might throw off it's algorithms, which is why we need to preprocess the individual light prior to stacking for dust motes (flats), hot pixels (dithering) and dark current wells (darks) as well as amp glow/ well irregularities (bias).  We might also try to do SOME background mitigation using PixInsight PixelMath and DBE manipulations at this point (I'm reluctant to remove raw data since I can never put that back), but the idea is that each individual light has as much non-uniform data removed from it as possible prior to doing the full stack.  Note that these manipulations have absolutely nothing to do with exposure time calculation ... these are to keep the stacking app happy ...

As part of this "quest for uniformity" we need to make the sky noise as uniform as possible; that's why we don't mix lights taken from an SQM 21 site with lights taken at an SQM 16 site in our stack ... although PixInsight can flatten noise pretty well with DBE and PixelMath, it's easier just not to have that skyfog noise variation there in the first place.  With this knowledge, and with the desire to keep this sky noise as constant as possible between lights and with the ability to get an idea (with my SQM meter) of what the SQM is each evening I simply use that to do a sorting of the lights by SQM gradation ...

And you're correct, this has nothing to do with transparency (which is seeing), and I can get that from the FWHM of the guide star for that evening; that brings in a whole new universe of target list sorting I do:

- If the calculated seeing is .8 or less, then I image all of my fine detail objects (small galaxies, planetary nebulae), those are placed in target list "Site A Fine"
- If the calculated seeing is .8 to 1.5, then I image all of my coarse detail objects (reflection nebulae, larger galaxies), those are placed in target list "Site A Coarse"
- If the calculated seeing is 1.5 to 2.0, then I image all of my binned stuff (generally larger nebulae and larger galaxies), those are placed in target list "Site A Binned"
- If the calculated seeing is 2.0 or larger (rare, but it happens) then I run my HyperStar objects, generally really big nebula like Veil, those are placed in target list "Site A HyperStar"

4. Regarding meridian flips. I think I understand but I want to ask to be sure. Is your goal that all images be taken on the same side of the pier for a given project? Or is it such a pain to do a meridian flip because of refocusing or whatever, that you want to do as few as possible on a given night?

For a given night I don't want to do ANY meridian flips for that evening ... the issue is that for larger SCT's like my C14's and C11's mirror flop is a significant issue when the scope transits the zenith and the mirror/ focuser tube "flop" from one side of the baffle tube to the other ... other than causing an image shift (which platesolving can fix) that will throw off the collimation of the OTA to some extent and now I've got fuzzy lights ... there are lot's of threads about this on CN and lot's of fixes, but each "fix" has it's own issues and problems that they introduce.  The best solution that I've found is to "just say no" to mirror flop in the first place by not letting the OTA transit the meridian.  When I collimate the OTA it's orientated E or W and is so marked, and I'm careful not to let the OTA "flop" from that orientation during transport or storage so that my collimation won't change.

Refocusing is easy (I use FocusMax and/or SGP), and I refocus before each light anyway to compensate for thermal changes ... it's the mirror flop and the loss of fine collimation I'm avoiding ...

5. I have the beginnings of an idea in mind to schedule in such a way as to limit the number of meridian flips. It works on the assumption that your goal is to minimize the number of flips during the night. So based on that assumption, how many times do you normally flip during the night? How do you decide when to do the flip?

The goal is "no flips period", and by creating two locations, each with an obstructed horizon, it's easy to do that ... location "E" has the horizon obstructed from 175deg to 360, location "W" has the horizon obstructed from 0 to 185deg, done and done.  I have "locations" stored for each site A based on pier side and SQM; A E SQM 21, A W SQM 21, A E SQM 20.8, A W SQM 20.8 and so on down to the last SQM.  Each "location" is set with the appropriate obstructed horizon and SQM level.

Thanks!

So, in summary here's what I've got so far for my workflow that I'm experimenting with using ST4:

- Once I've determined the seeing for the site I'll load in the appropriate target list in the "Target Selection" tab, say that I think it's going to be a murky 1.5 seeing day so I load in the "Site A Coarse" target list 
- I take the last night SQM reading, say it's 20.5, so I load in the location "Site A SQM 20.5 E", set my imaging and telescope system as appropriate (I have C14's and C11's) and generate my schedule for the E pier scopes
- I then load in the location "Site A SQM 20.5 W" and generate my schedule for the W pier scopes
- The SGP's controlling the E scopes get the E schedule, the SGP's controlling the W scopes get the W schedule ...

SGP on each scope does it's thing, hopefully properly ...

- During the day I take my flats for each scope
- I combine the flats for that scope, the lights it took that evening, a master dark for the temperature and that scope sensor into a zip file labeled for that date/ SQM/ scope/ FOV and file it away
- Rinse, lather, repeat day after day, week after week, month after month ...

Eventually I have enough light zips accumulated for an object (generally 50+) by scope type, SQM, and FOV that I unpack everything and stack it all up with various stacking apps, if I like what I see then the result goes into PixInsight/ APP/ whatever ...

Hope this gives a bit more clarity!

[theskyhound]

Hi,

How on earth are you scheduling from the target list? Only projects can be scheduled. The Target Selection Tool is used to select targets, which become projects. It is projects that you are scheduling, so I don't see how the target list helps you schedule. I'm confused.

Ok, I see what you mean by detail, but that's not really what it is. This represents the signal that you are exposing for. It is what the SNR is computed for. Detail is about resolution.