HowToUseImprovedPointingProducts < Public

Public Web>HowToUseImprovedPointingProducts (2013-05-22, MigoMueller) (raw view)
---+ New Pointing Products for ODs 320 to 761

%TOC{depth="2"}%

<h3><span style="text-decoration: blink;"><b>Important:</b> </span></h3> if the data retrieved from the Herschel Science Archive (HSA) have been processed with SPG V9.1.0 or later (check the "creator"  metadatum within the observation context), the improved pointing corrections are already in the observation context and therefore you *DO NOT* need to apply this procedure


---++ Introduction

On OD 320 the temperature of the star-tracker (STR) CCD was lowered, to mitigate the effect of hot flickering pixels. As a side effect this change slightly the focal length of the star tracker, hence introducing a shift in the reconstructed astrometry of Herschel maps when the distribution of tracking stars was not homogeneous in the STR field-of-view, up to 7-8 arcsec in the most extreme cases. A new model was uploaded on-board on OD762 to correct for this. For the interim period ODs 320 to OD761 new first order pointing products have been generated at HSC to correct this problem.

Below an example for a star (UZ Tau) in OD684, on the left the current HSA level 2 blue channel (70 microns) maps and in the right processed with the new pointing product. The absolute astrometry offset is reduced from 7.5 arcsec to 1 arcsec.<br />

More details are provided in the [[%ATTACHURLPATH%/new_pointing_products_readme_v1p0.pdf][Release Note v1.0]]

A copy of the test report can be found here [[%ATTACHURLPATH%/New_Pointing_Products_Test.pdf][New pointing products test report (PDF)]]


 <img src="%ATTACHURLPATH%/UZ_Tau.jpg" alt="UZ_Tau.jpg" width='416' height='274' />

Here is a SPIRE example: PSW map GammaDra from OD450 with/without improved pointing correction: although from the circle looks like there isn't any significant offset change, it is possible to appreciate the (tiny) difference in flux distribution in the pixels.<br />

 <img src="%ATTACHURLPATH%/GammaDra_450_PSW.png" alt="GammaDra_450_PSW.png" width='406' height='365' />

The new pointing products are posted on 
ftp://ftp.sciops.esa.int/pub/msanchez/corrected_pointing_od320_761/


---++ How to use the new pointing product for PACS photometer scan maps
Replace the line extracting the pointing product :
<verbatim>
pp = obs.auxiliary.pointing
</verbatim>
by:
<verbatim>
pp = fitsReader('/my_directory_path/pointing_od_0nnn.fits')
</verbatim>
where nnn is the OD number

Note that if you use the SPG pipeline scripts, the pointing product is read twice in L05_phot.py and L1_scanMapPhotProject.py, hence the line must be commented out in the second script.

---++ How to use the new pointing product for PACS spectrometer observations
Load the new pointing product :

<verbatim>
pp = fitsReader('/my_directory_path/pointing_od_0nnn.fits')
</verbatim>
where nnn is the OD number

You need to replace <verbatim>obs.auxiliary.pointing</verbatim> in two pipeline tasks:

<verbatim>
slicedFrames = specAddInstantPointing(slicedFrames, obs.auxiliary.pointing, calTree = calTree, orbitEphem = obs.auxiliary.orbitEphemeris, horizonsProduct = obs.auxiliary.horizons)    
# Replace with:
slicedFrames = specAddInstantPointing(slicedFrames, pp, calTree = calTree, orbitEphem = obs.auxiliary.orbitEphemeris, horizonsProduct = obs.auxiliary.horizons)
</verbatim>

and 

<verbatim>
slicedFrames = specCorrectHerschelVelocity(slicedFrames, obs.auxiliary.orbitEphemeris, obs.auxiliary.pointing, obs.auxiliary.timeCorrelation, obs.auxiliary.horizons)
# Replace with:
slicedFrames = specCorrectHerschelVelocity(slicedFrames, obs.auxiliary.orbitEphemeris, pp, obs.auxiliary.timeCorrelation, obs.auxiliary.horizons)   
</verbatim>

    
---++ How to use the new pointing product for SPIRE observations

This should be used for observations executed from OD320 to OD761 inclusive. The OD number can be checked in the observation context metadata =print obs.odNumber=. Replace the line in the user reprocessing script or the SPG script that extracts the pointing product from the observation context:
<verbatim>
hpp = obs.auxiliary.pointing
</verbatim>
by:
<verbatim>
hpp = fitsReader('/my_directory_path/pointing_od_0nnn.fits')
</verbatim>
where nnn is the OD number. This approach is applicable for both the SPIRE photometer and the SPIRE spectrometer processing.

---++ How to use the new pointing product for HIFI observations

The new pointing products need be applied to the level0 products. The following code indicates how to reprocess a given observation with the new astrometry. Note that you can restrict yourself to the backends you want to update, however be aware that the ra/dec meta-data keywords of the observation context will be based on the average of the level2 products of all spectrometers, so it will reflect the astrometry shift only if all backends are fully reprocessed.

<verbatim>
############################################################
from herschel.hifi.pipeline.level0 import DoPointingTask
doPointing=DoPointingTask()
#
#Copy the new pointing product pool and read it in HIPE - nnn is the OD number
mypointing = simpleFitsReader('/my_directory_path/pointing_od_0nnn.fits')
#
obs = getObservation(obsid,useHsa=True)
lev0=obs.getProduct("level0")
#
from herschel.hifi.pipeline.generic.utils import ChopperPosition
ChopperPosition.initialize( obs.calibration, obs.getStartDate() )
chopperThrow=obs.calibration.getCalNode( "Downlink" ).getCalNode( "Generic" ).getProduct("chopperThrows").getByStartDate(obs.getStartDate(),"ChopperThrows").getProduct().getTable()
#
for htpname in ["WBS-H", "WBS-V","HRS-H","HRS-V"]:
  htp=lev0.getProduct(htpname)
  htp=doPointing(htp=htp, useIntegration=1, aux= obs.auxiliary, pointing=mypointing,chopperThrows=chopperThrow)
#
obs=hifiPipeline(obs=obs, apids=["WBS-H", "WBS-V","HRS-H","HRS-V"],fromLevel=0, upToLevel=2.5)
############################################################
</verbatim>

The following plot illustrates the case where the largest shift was observed (obsid 1342203210): the blue dots show the position with the old pointing products, while the red dots show the new position. It is also possible that the position shift is not homogeneous over a given map (see example for obsid 1342207386), so if you have large maps please check carefully whether this can be the case.

<img src="%ATTACHURLPATH%/AllFrameComparison_1342203210_twiki.png" alt="AllFrameComparison_1342203210_twiki.png" width='406' height='365' />
<img src="%ATTACHURLPATH%/AllFrameComparison_1342207386_twiki.png" alt="AllFrameComparison_1342203210_twiki.png" width='406' height='365' />

---++ README information - first-order corrected pointing products

MSP, 27th July 2012 v1.0

---+++ Background information

A main component of the Attitude Control and Measurement System (ACMS) is the star-tracker (STR; two units present on-board Herschel, prime and backup). The STR is a small optical CCD camera plus an image processing unit that, starting from an image of the sky, extracts the attitude information measured with respect to the J2000 inertial reference system and delivers it to the Attitude Control Computer (ACC). At the start of the mission, the STR CCD reference temperature was 20 deg C. An effect that was observed along the first months of S/C operations was the sporadic occurrence of distortions in the scan map patterns accompanied by variations in the tracking angular rate (the so-called 'speed bumps'). These events were eventually tracked to the existence of transitory 'warm pixels' (not flagged as 'hot' ones in the defective pixel table). In order to reduce the number of warm pixels, the STR CCD reference temperature was lowered in operational day (OD) 320 from 20 to -10 deg C. The 'speed bumps' were eventually corrected, but at the price of augmenting the STR plate scale errors that produced systematic but boresight-dependent offsets. 

The new (low) temperature setting had a negative impact on the S/C absolute pointing accuracy due to increased plate scale errors in STR. In OD 762, a simple, 1D correction (i.e. the same linear correction factor in the Y and Z STR axes) was uploaded to the S/C ACMS and the performance was substantially improved. In OD 858 an enhanced 2D correction (different linear factors in the Y and Z axes) was successfully tested; the change was made permanent in OD 866. Eventually, in OD 1011, a full STR focal plane (FP) distortion correction was made operational. Moreover, in OD 1032 the STR catalogue was purged by removing 73 'dubious' stars from the tracking list. 

Several groups within the Herschel Ground Segment are currently working in the a posteriori improvement of the S/C attitude, coordinated by the Herschel Pointing Working Group. This is specially important for observations obtained in the period OD 320 - OD 761, as explained above. 

The pointing products produced by the software described in this note intend to provide a first, yet crude correction of the systematic, boresight-dependent effects introduced by the STR plate scale errors.


---+++ Scope of applicability

The current version of the software is mainly intended to alleviate the problems derived from the large STR plate scale errors existing in the period OD 320 - OD 761. A correction has been computed for the period OD 762 - OD 865 (excluding OD 858), but its actual usefulness is still under evaluation and no new pointing products have been made available in the ftp area for that range of ODs.

If your observation has been gathered in OD 858 or OD >= 866, no correction is required (since either the 2D or the full STR FP correction has been already applied). 

Finally, for ODs before 320, no STR FP correction is available yet but it is expected in a near future. As soon as it is included in the processing software, new pointing products will be generated. Nevertheless, it is advisable to use it only in extreme cases, where large pointing offsets are suspected.   

It is intended to use the new pointing products in the upcoming (v9.x, TBD) HSA bulk reprocessing. Therefore, no pointing product replacement will be required for products retrieved from the HSA processed with SPG v9.x.

---+++ Processing steps

The current processing software applies the following steps:

<verbatim>
(a) Load the STR guide stars' catalogue 
(b) Open the pointing product and read records in a loop. For each attitude sample:
   b.1 Read filtered attitude and STR quality flag (number of stars tracked)
   b.2 Get the catalogue information fro stars within the FoV of the STR at the current S/C filtered 
       attitude: X,Y,Z, magnitude, and trackability status.
   b.3 For trackable the stars within the FoV, determine the direction vectors using the
       STR "focal length" correction factors provided by Flight Dynamics.
   b.4 Determine the tracked stars by means of the same algorithm used in the STR (but limited to a maximum         
   of nine stars, i.e. the "interlacing mode" is not implemented) 
   b.5 For the tracked stars, given the direction vectors and the inertial
      (reference) vectors, determine the best attitude using the "q-Method" (Davenport 1968).
   b.6. Replace the filtered quaternion by the modified quaternion computed by the q-method.
</verbatim>

The applied correction does not add noise to the attitude samples since is a purely geometric one. When the correction factors are set to 1.0, one recovers exactly the input attitude (within ~1.0e-10 arcsec).

In the current implementation, the processing script is not distributable to end- users since it requires the proprietary STR catalogue.


---+++ Limitations 

<verbatim>
(a) Only up to 9 stars are used, i.e. no interlacing is considered
(b) The stars' selection algorithm uses the same logic as the STR. But there is no check that the stars used within the script are the same as those actually used for tracking. In particular, if one of the 73 bad stars was used for tracking, it will replaced by a different one in the script.
(c)  The 2-D, linear correction is implemented (rather than the full one).
</verbatim>

---+++ Future improvements

The software described here is just a first step in a long-term strategy to improve the S/C attitude estimation and hence the astrometry of the science products. It is expected to have further releases of the current software that will include things like:

- New coefficients for the early phases of the mission
- New STR star selection algorithms based on the guide stars _actually_ selected by the ACMS
- Implementation of the interlacing algorithms
- Implementation of the STR full FP correction

Future software tools will include a more sophisticated processing based on batch attitude estimators. The construction of such software is currently being started. 

---+++ Final remark

Do not apply the new pointing products blindly. If you are happy with the current astrometry of your observations, it is probably not worth to reprocess the data. In case of doubt, contact the HSC Helpdesk for advice.

---++ STR-induced offsets for observations in the period OD320-761

The files linked below list the expected STR-induced offsets at the nominal RA, DEC coordinates (i.e. those entered as "target" coordinates in HSPOT) for all observations in the range OD 320-761, in comma-separated format (readable in Excel, Open Office, TOPCAT and many other common tools).

   * [[%ATTACHURL%/offsets_HIFI.csv][offsets_HIFI.csv]]: STR-induced offsets at the nominal RA, DEC positions for HIFI observations in the range OD320-761

   * [[%ATTACHURL%/offsets_PACS_Photometer_od320-550.csv][offsets_PACS_Photometer_od320-550.csv]]: STR-induced offsets at the nominal RA, DEC positions for PACS Photo observations in the period OD320-550

   * [[%ATTACHURL%/offsets_PACS_Photometer_od551-761.csv][offsets_PACS_Photometer_od551-761.csv]]: STR-induced offsets at the nominal RA, DEC positions for PACS Photo observations in the period OD551-761

   * [[%ATTACHURL%/offsets_PACS_Spectrometer_RangeSpec.csv][offsets_PACS_Spectrometer_RangeSpec.csv]]: STR-induced offsets at the nominal RA, DEC positions for PACS Spectrometer RANGE observations in the period OD320-761

   * [[%ATTACHURL%/offsets_PACS_Spectrometer_LineSpec.csv][offsets_PACS_Spectrometer_LineSpec.csv]]: STR-induced offsets at the nominal RA, DEC positions for PACS Spectrometer LINE observations in the period OD320-761

   * [[%ATTACHURL%/offsets_SPIRE_Photometer_od320-550.csv][offsets_SPIRE_Photometer_od320-550.csv]]: STR-induced offsets at the nominal RA, DEC positions for SPIRE Photo observations in the period OD320-550

   * [[%ATTACHURL%/offsets_SPIRE_Photometer_od551-761.csv][offsets_SPIRE_Photometer_od551-761.csv]]: STR-induced offsets at the nominal RA, DEC positions for SPIRE Photo observations in the period OD551-761

   * [[%ATTACHURL%/offsets_SPIRE_Spectrometer_od320-761.csv][offsets_SPIRE_Spectrometer_od320-761.csv]]: STR-induced offsets at the nominal RA, DEC positions for SPIRE SPEC observations in the period OD320-761

-- Main.MiguelSanchez - 07 Sep 2012
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	AllFrameComparison_1342203210_twiki.png	r1	manage	26.1 K	2012-09-07 - 10:14	DavidTeyssier
png	AllFrameComparison_1342207386_twiki.png	r1	manage	30.5 K	2012-09-07 - 10:33	DavidTeyssier
png	GammaDra_450_PSW.png	r1	manage	420.1 K	2012-09-03 - 15:40	LucaConversi	PSW map GammaDra from OD450 with/without improved pointing correction
jpg	UZ_Tau.jpg	r1	manage	22.3 K	2012-07-24 - 16:25	BrunoAltieri	UZ Tau new astrometry OD684
pdf	new_pointing_products_readme_v1p0.pdf	r1	manage	60.6 K	2012-09-07 - 09:46	AnthonyMarston	ReadMe for pointing products -- 7 Sep 2012
pdf	New_Pointing_Products_Test.pdf	r3 r2 r1	manage	1673.3 K	2012-09-07 - 13:16	AnthonyMarston	New pointing product test report -- 7 Sep 2012
Topic revision: r21 - 2013-05-22 - MigoMueller
Public
Webs
Public
TWiki