Herschel Open Time Cycle-1 Data Processing Workshop, 14-18 March 2011

DO NOT FORGET TO FILL IN THE FEEDBACK FORM!

Feedback form

We want to improve! Please send us your feedback using the form below:

http://www.surveymonkey.com/s/Herschel_OT1_DPWS_Feedback

Thanks!

Final Agendas

Sessions will start on Monday 14 March at 09:30 and at 09:00 on the rest of the days of the workshop. The PACS instrument part of the workshop will end at around 13:30h on Wednesday while the HIFI & SPIRE sessions will start at 14:30h on the same day. The workshop will end on Friday at 16:15h.

• OT1_PACS_DPWorkshop_final_agenda.pdf: OT1_PACS_DPWorkshop_final_agenda.pdf
• OT1_HIFI_SPIRE_DPWorkshop_final_agenda.pdf: OT1_HIFI_SPIRE_DPWorkshop_final_agenda.pdf

List of participants

• List of Participants (PDF)

Presentations

Monday 14 March 2011

• Day1_overview_Herschel_Pilbratt.pdf: (G. Pilbratt)
• Day1_overview_instr_Marston.pdf: (A. Marston)
• Day1_overview_DP_Ott_v2.pdf: (S. Ott)
• Day1_HSA_User_Interface_Verdugo.pdf: (E. Verdugo)
• Day1_Data_Access_Verdugo.pdf: (E. Verdugo)
• Introduction_to_HIPE.pdf: (B. Merín)
• Introduction_to_HIPE_D.Shupe_NHSC_DPWS.pdf: (D. Shupe)
• HIPE documentation (D. Rizzo)
• Day1_PACS_Photometer_MSauvage.pdf: (M. Sauvage)
• 2011_March_DP_workshop_phot_level0_1.pdf: PACS photo from level0 to level1 (B. Altieri)
• 2011_March_DP_workshop_phot_level1_2.pdf: PACS photo from level1 to level2 (B. Altieri)
• PACS_Phot_Deglitching.pdf: Deglitching PACS photometry data (N. Billot)

Tuesday 15 March 2011

• Day2_jython_scripting_msp_110314.pdf: Scripting in HIPE (M. Sánchez-Portal)
• Day2_PACS_photometer_calibration.pdf: (B. Altieri)
• Day2_PACS-MADmap.pdf: (M. Sauvage)
• Day2_scanamorphos_workshop_mar11.pdf: (H. Roussel)

• Day2_PACS_Spectrometer_Introduction.pdf: (B. Vandenbussche)
• Day2_PACS_Spectrometer_Products.pdf: (R. Vavrek)

Wednesday 16 March 2011

• Day3_overview_DP_Ott.pdf: (S. Ott)

Thursday, 17 March 2011

SPIRE

Time	Presentation	Presenter	Notes, Scripts etc.
09:15	SPIRE Observing Modes AOT reminder and SPIRE status	C. Pearson
09.35	SPIRE User Documentation	B. O'Halloran
09:45	Reveiw of SPIRE set up for Workshop	C. Pearson	script
10:00	Tour of SPIRE data products and Quick-Look Tools in HIPE	C. Pearson	DRG Chapter 2
11:30	SPIRE calibration trees	I. Valtchanov	DRG Chapter 3
11:50	SPIRE Photometer Pipeline (Do I need to re-process my data?)	C. Pearson
12:20	SPIRE Spectrometer Pipeline (Do I need to re-process my data?)	E. Polehampton
14:30	SPIRE demo: how to get data into SpectrumExplorer	E. Polehampton
15:30	Line fitting in SpectrumExplorer using a script	E. Polehampton	script and line list

HIFI

Time	Presentation	Presenter	Notes, Scripts etc.
09:15	HIFI Observing Modes and Status	Russ Shipman
09:45	HIFI set up, unpack data, check installation	Carolyn McCoey	tgzfiles_to_lstore.py HipeInstallationCheck_demo.py
10:15	Tour of HIFI Observation	Russ Shipman	tour_of_hifi_data.py
11:30	HIFI Pipeline Overview	Carolyn McCoey
12:00	Demo/hands-on Reprocessing Observations and Customising the HIFI pipeline	Carolyn McCoey	command_line_pipeline.py modifiedPipelineDemo.py advanced_repipeline_demo.py
14:30	HIFI/SPIRE joint spectral tools sessions	Carolyn McCoey, Ed Polehampton	SpectrumExplorer (SPIRE) SpectrumExplorer_demo.py SpectrumFitter (SPIRE) spirelines.dat SpireSpectrumFitterDemo.py spectrumfitter_scripting_handson.py SpectrumTools_handson.py

• Day3_OT1_ESAC_DPwksp_Cassis.pdf: (S. Bottinelli)

Friday, 18 March 2011

SPIRE

Time	Presentation	Presenter	Notes, Scripts etc.
09:00	Reprocessing your photometer data with SPIA	C. Pearson	SPIA standard processing video tutorial
11:00	SPIRE Photometer Mapping Troubleshooting and Tips	C. Pearson	script; DRG Section 5.1.3
11:45	SPIRE Photometer deglitching, mask editing	L. Conversi	script; DRG Section 4.3
12:30	SPIRE Source Extraction & Aperture Photometry	B. O'Halloran	script; DRG Section 5.4
13:15	Considerations for Parallel Mode	L. Conversi	Combine script & Split script
14:30	Spectrometer reprocessing scenarios for (weak/medium strength) point sources	N. Lu	DRG Section 6.2.5.1
15:15	Spectrometer reprocessing scenarios for extended sources	I. Valtchanov	DRG Section 6.2.5.3

HIFI

Time	Presentation	Presenter	Notes, Scripts etc
09:00	HIFI Spurious Features	Carolyn McCoey	findSpurs_flagData_demo
09:30	HIFI standing wave and baseline removal	Sylvie Beaulieu	FitHifiFringe_demo.py
10:00	Baseline Fitting	Sylvie Beaulieu	See standing wave presentation FitBaseline_demo.py
11:00	HIFI Deconvolution	Ian Avruch	Supplementary image 1 ThesisJWK-21Nov2008.pdf" target="_top"> Supplementary image 2 handson_sscan.py
12:00	Free hands-on time		flagging_and_decon_handson.py
13:00	Saving / Importing and Exporting presentation	Russ Shipman	export_to_class.py
14:30	HIFI Mapping modes, cube creation, cube visualisation	Carolyn McCoey	mapping_demo.py MappingHandsOn.pdf mapping_handson.py

* HifiPresentations.zip: All HIFI Presentations

* HifiScripts.zip: All HIFI scripts

Data sets

To download the data sets that will be used during the workshop demos/hands-on sessions, please follow the instructions below:

ftp ftp.sciops.esa.int
Name: anonymous
Password: your email address
cd /pub/hsc_wshop
prompt
bin
dir

Then, change to the directory you are interested in and write:

mget *

You can also go to ftp://ftp.sciops.esa.int/pub/hsc_wshop/ with your favorite FTP compatible web browser.

You can place the downloaded data sets in any folder of your choice. You do not need to unzip or untar the files for the moment. We will later provide information at the workshop on how to move these zipped and tarred files into the appropriate area for HIPE to recognize them.

IMPORTANT INSTRUCTIONS FOR PACS SESSIONS PARTICIPANTS

If you are to participate in the PACS hands-on sessions you will need to start up HIPE 6.0 to allow the software to update and construct automatically the calibration files. Earlier versions of HIPE handle the calibration files differently, so this step is very important. When the software is started up for the first time it will ask you to update the calibration. You should accept the update and allow some time for the calibration files to be copied to your system. Once this is done, you may close HIPE. It is mandatory to perform this step the first time HIPE 6.0 is launched.

IMPORTANT INSTRUCTIONS FOR HIFI/SPIRE SESSIONS PARTICIPANTS

ALL HIFI/SPIRE workshop participants should download

HIFI data 1342205474_map_dbs_1b.tgz

and

SPIRE data spireSpec1342189124_NGC7027.tgz

These datasets will be used in the joint HIFI/SPIRE spectral Tools session and are placed in the corresponding HIFI or SPIRE directories in the ftp repository above mentioned.

PACS Data sets

PACS Photometry

NOTE: for the RCW120 observations only the red band can be processed on a 4 GB RAM laptop. All public data in the archive (Total size= 0.77GB)

OBSIDs	Target	description
1342185553	RCW120	scan map
1342185554	RCW120	scan map
1342185575	Sh104	scan map
1342185576	Sh104	scan map
1342189191	delta Dra	mini-scan map
1342189192	delta Dra	mini-scan map
1342195466	HIP99240	mini-scan map
1342195467	HIP99240	mini-scan map

PACS Spectroscopy

OBSIDs	Target	description
1342191353	AFGL490	Line spectroscopy
1342204332	Ceres	SED
1342204333	Ceres	SED
1342197900	Neptune B2B ON	Unchopped SED
1342197901	Neptune B2B OFF	Unchopped SED
1342197898	Neptune B2A ON	Unchopped SED
1342197899	Neptune B2A OFF	Unchopped SED
1342209696	NGC6543	Unchopped Line spectroscopy
1342209695	NGC6543	Unchopped Line spectroscopy

HIFI Data sets

NOTE: Total size: 4.28GB

OBSIDs	AOR Label	Note	Pool name used in scripts
1342180473	Aot2_P_FSw_1b_H2O_557_AFGL2591_deep		1342180473_point_fsw_1b
1342180551	Aot2_P_LChopNoRef_3a_CH+_DR21		1342180551_point_lchop_3a_noref
1342181161	Aot2_S_DBS_1b_full_R4_LDN1157B1		1342181161_sscan_dbs_1b_r4
1342190183	Aot2_P_FastDBS_1b_CO5-4_LDN1157		1342190183_point_fastdbs_1b
1342191505	SDP_HOP - HScan - O2 - 487.3GHz - Orion S		1342191505_sscan_mini_dbs_1a_r12
1342191649	SDP_HEXOS-orikl-hifiss-b4b		1342191649_sscan_dbs_4b_r6
1342205481	Aot3_M_DBS_3x3_7b_CII_S140	Engineering mode	1342205481_map_dbs_7b
1342205474	Aot3_M_DBS_3x3_1b_CO_5_4_S140	Engineering mode	1342205474_map_dbs_1b
1342210099	Aot3_M_OTF_1b_CO5-4_IRC10216		1342210099_map_otf_1b
1342210097	Aot3_M_OTF_1b_CO5-4_IRC10216_halfbeam		1342210097_map_otf_1b_halfbeam

SPIRE Data sets

• Description of the SPIRE Data Pools: SPIREDataPools.pdf

OBSIDs	Description	Source	Pool Name
1342210941	Small Map of planetary nebula	BD303639	spirephot_bd303639
1342211401	Small Map of “dark region”	Dark Sky	spirephot_dark_od587
1342195871	Small Map of secondary calibration star	GammaDra	spirephot_gammadra_rp
1342187440	Large Map of Neptune, OD 202	Neptune	spirephot_neptune_beams_rp
1342183475	Large Map of planetary nebula	NGC 5315	spirephot_ngc5315_rp
1342183046	Parallel Mode maps of bright galaxy (orthogonal direction only)	NGC 6946	spirephot_ngc6946_ort_rp
1342183047	Parallel Mode maps of bright galaxy (nominal direction only)	NGC 6946	spirephot_ngc6946_norm_rp
1342189124	Carbon star NGC7027 (OD240, 17 reps, HR mode)	NGC 7027	spireSpec1342189124_NGC7027
1342189120	Dark sky (OD240, 40 reps, CR mode)	Dark Sky	spireSpec1342189120_DarkSky
1342187893	Mrk 231 (OD209, 50 reps, HR mode)	Mrk 231	spireSpec1342187893_Mrk231
1342187890	Dark sky (OD209, 50 reps, CR mode)	Dark Sky	spireSpec1342187890_DarkSky
1342192180	NGC7023 (OD302, 4 reps, MR, Raster, Intermediate sampling)	NGC 7023	spireSpec1342192180_NGC7023

The spectrometer reprocessing demos will also make use of the following two calibration files (already in the ftp delivery above):

• SCalSpecTeleRsrf_0x50002972_HR_UNAPOD_v6_1.fits
• SCalSpecTeleRsrf_0x50002E40_HR_UNAPOD_v6_1.fits

Scripts

General

• Scripts to load the test data for this DP workshop:

• tgzfiles_to_lstore_pacsphot.py.txt: tgzfiles_to_lstore_pacsphot.py.txt -- unpacks the data for this workshop

• tgzfiles_to_lstore_pacsspec.py.txt: tgzfiles_to_lstore_pacsspec.py.txt -- unpacks the data for this workshop

• tgzfiles_to_lstore_spirephot.py.txt: tgzfiles_to_lstore_spirephot.py.txt

• tgzfiles_to_lstore_spirespec.py.txt: tgzfiles_to_lstore_spirespec.py.txt

• tgzfiles_to_lstore_hifi.py.txt: tgzfiles_to_lstore_hifi.py.txt

• Other demo scripts and needed auxiliary files:

• jythonScriptExample.py.txt: jythonScriptExample.py.txt -- example used in the "Scripting in HIPE" session

• process_program.py.txt: process_program.py.txt (needs obsid_spirephot.txt)

• obsids_spirephot.txt: obsids_spirephot.txt

PACS

• scanmap_Deep_Survey_miniscan_Pointsource.py.txt: scan map deep survey mini-scan point-source
• multiple_obsid_scanmap_Deep_survey_miniscan_Pointsource.py.txt: Multiple obsid's mini-scan map point-source

• scanmap_Extended_emission_PhotProject.py.txt: Extended source scan map
• multiple_obsid_scanmap_Extended_emission.py.txt: Multiple obsid's scan map extended source

• L25_scanMapMadMap.py.txt: MADmap script for crossed maps
• L25_scanMapMadMapMod.py.txt: MADmap script for crossed maps (workshop version presented by M. Sauvage)

• plotscanpath.py.txt: plot scan path

• DP_Workshop_PACS_Spectro_PierreRoyer.py.txt (workshop version by P. Royer, it needs HIPE 7.0 to run)

• ExamplePlots_PRoyer_Demo.zip: ExamplePlots_PRoyer_Demo.zip (plots generated by P. Royer's demo)

SPIRE

• tgzfiles_to_lstore_spire.py.txt: SPIRE data unpacking script

• SPIRE_photometer_baseline.py.txt: SPIRE_photometer_baseline.py.txt

• spirelines.dat.txt: spirelines.dat.txt

• SpireSpectrumFitterDemo.py.txt: SpireSpectrumFitterDemo.py.txt

• OT1_Workshop_SPHOT_combine_obs.py.txt: OT1_Workshop_SPHOT_combine_obs.py.txt

• OT1_Workshop_SPHOT_remove_glitches.py.txt: OT1_Workshop_SPHOT_remove_glitches.py.txt

• SPIRE_Photometry_GammaDra.py.txt: Source extraction script

HIFI

• HifiScripts.zip: All HIFI scripts

Special Hands-on Sessions

SPECIAL HANDS-ON SESSION ON CASSIS

On Thursday March 17 (16:45h) a presentation of the CASSIS software package as a HIPE Plugin will be given by Sandrine Bottinelli. On Friday March 18 (12h-13h, room B65), a hands-on session on CASSIS supervised by Jean-Michel Glorian (CASSIS developer), Sandrine Bottinelli and Emmanuel Caux (CASSIS Scientists) is organized. This hands-on session will make use of the HIFI, PACS and SPIRE data provided for the workshop.

The CASSIS Plug-in can be installed in your HIPE 6.0.3. CASSIS is a spectral analysis tool-set containing:

• Line Identification Tool (via a CASSIS internal database containing the JPL, CDMS and NIST databases) • Prediction of spectra (HIFI only at this time) • Powerful line/baseline fitting • Comparison of telescope or laboratory data with a wide variety of models • Tools to determine the physical parameters of the sources

See the cassis web-site for details (http://cassis.cesr.fr).

The plug-in makes these functionalities available directly in HIPE.

To install the CASSIS plugin, select "Plug-ins" from the HIPE Tools menu. Click "Install new plug-in" and enter the following URL:

http://cassis.cesr.fr/download/cassis_2.8.jar

The plugin has been successfully tested with HIPE 6.0.3 within the following environments :

- Mac OSX 10.6 Snow Leopard - Windows XP and Windows 7 - Linux and Unix

It would be better for the users who plan to attend the CASSIS hands-on to come to the workshop with the CASSIS plugin installed on their laptops in HIPE 6.0.3 as the download and install of the plugin can take a while if done simultaneously from ESAC (~350 Mb to download).

Booklets

• Discover HIPE Help (PDF)

Video Tutorials

• Learn HIPE YouTube channel - Video Tutorials
• NHSC PACS DP Video Tutorials