1

GeneratingLightcurvesfromForcedPSF-fitPhotometryonZTFDifferenceImages

FrankMasci1andRussLaher2

August1,2019,v1.8

Outline1–Introduction......................................................................................................................................p.12–Beforeyousubmityourrequest..............................................................................................p.23–Howtosubmityourrequest......................................................................................................p.34–WhocansubmitarequestandhowwillIbenotified?..................................................p.35–Rawoutputtable:whatdoesitmean?..................................................................................p.46–Advisoriesandcautionarynotes.............................................................................................p.77–Exitandwarningcodes(theprocstatuscolumn).............................................................p.88–Baselinecorrectionusingtherawfluxlightcurveofasuspecttransient..............p.99–Validatingphotometricuncertaintiesandpossiblerescaling....................................p.1110–Generatingdifferential-photometrylightcurveswithnon-detections.................p.1211–Generatingabsolute-photometrylightcurvesforvariablesources.......................p.1212–Goingdeeper:combiningsingle-epochmeasurements..............................................p.14

1IntroductionThisdocumentdescribeshowtosubmitaforcedphotometryrequesttooperateonarchiveddifferenceimagesfromtheZTFsurvey.ThesedifferenceimageswouldhavebeeninitiallygeneratedbyIPAC’sreal-timeproductionpipeline.Wealsocoverthefundamentalsofgeneratingpublication-qualitylightcurvesfromtherawoutput,aswellascaveats,warnings,andsuggestionsforoptimizingthem,forexamplebycombiningmeasurementsinordertoplacetighterconstraintsorrecoversignalbelowthesingle-exposurelimits.Weadviseyoufamiliarizeyourselfwiththecautionarynotesin§2and§6.Also,bepreparedtowriteascripttoanalyzetherawtableoutputandgeneratelightcurves.Werecommendvisualinspectionoftheimages,withguidancefromthevariousepoch-basedfiltersintherawoutput.Staytunedformoreadvisoriesaswelearnthem,primarilyfromyou.Ifyourpublicationusesproductsfromthisservice,werecommendreferringtothisas:“TheZTFforced-photometryservice…”andcitingthefollowing:

Masci, F. J., Laher, R. R., Rusholme, B., et al. 2018, The Zwicky Transient Facility: Data Processing, Products, and Archive, PASP, 131, 995.

Also, if your publication used this service, please include the following in the acknowledgments: The ZTF forced-photometry service was funded under the Heising-Simons Foundation grant #12540303 (PI: Graham).

1fmasci@caltech.edu2laher@ipac.caltech.edu

2

2BeforeyousubmityourrequestHerearesomeadvisoriestoavoidpotentialstressforbothyouandthemachinethatwillbeprocessingyourrequest.(a) Ensureyourtargetofinteresthasbeenvettedbyothermeans,i.e.,thereisindependent

confirmationthatitisrealorpotentially“interesting”.Thisgoeswithoutsaying,right?Basically,theforced-photometryserviceisnotavettingservicefordisseminatingcandidatesdiscoveredfromtheindividualepochalimages.

(b) EnsureyoursuppliedR.A.,Dec.positionisaccurateandreflectswhatwouldbemeasureddirectlyoffIPAC’sastrometrically-calibratedscienceimagesinthearchive.Thepositionbasedonasinglediscoveryepochmaynotbeaccurateenough.

(c) Ensureyourequestasufficientnumberoffutureand/orhistoricalepochsafter/priortotheJulianDate(JD)definingyourtransienteventorcharacteristicbehavior.Wesuggest≥30additionalepochs.Thepurposeistoenableapossiblebaselinecorrection(Section8)and/oruncertaintyrescaling(Section9).Thiswillalsoallowyoutocombinethesingle-epochmeasurementsinordertogodeeper(Section12).Ensuringadditionalfuture/historicalepochsisusuallyunnecessaryforcontinuous(a)periodicvariables.Itisassumedyouunderstandyourtargetwellenoughtojudiciouslyselectthetime-span.

(d) Pleasedonotsubmitmorethan100individualtargetsinonego.Youwillbecompetingwithothersforresources.Pleasedonotsubmitanotherbatchoftargetsuntilyouhavebeennotifiedthatallrequestsinyourprevioussubmissionarecompleteandreadyfordownload.

3

3HowtosubmityourrequestAnindividualsubmissionconsistsofsupplyingasingleR.A.,Dec.positionandtime-rangeofinterestusingthewgetcommand-lineutility.Belowisasynopsis,wherequantitiesintheangledbracketsdenoteuserinputs.wget --http-user=<UNAME> --http-passwd=<PASSWD> “https://ztfweb.ipac.caltech.edu/cgi-bin/requestForcedPhotometry.cgi? ra=<RA>&dec=<DEC>&jdstart=<JD_START>&jdend=<JD_END>&email=<EMAIL_ADDRESS>” where:UNAME=Usernametoaccesstheservice–pleasecontactfmasci@caltech.eduPASSWD=Passwordtoaccesstheservice–pleasecontactfmasci@caltech.eduRA=RightAscensionofyourtargetposition[decimaldegrees;J2000]DEC=Declinationofyourtargetposition[decimaldegrees;J2000]JD_START=JulianDatecorrespondingtoearliestepochofsingleexposure[decimaldays]JD_END=JulianDatecorrespondingtolatestepochofsingleexposure[decimaldays]EMAIL_ADDRESS=YourloginaddresswhenaccessingproductsfromtheZTFArchive3Example(tobeexecutedasasinglelineonyourcommand-lineprompt):wget --http-user=xxxx --http-passwd=xxxx -O log.txt "https://ztfweb.ipac.caltech.edu/cgi-bin/requestForcedPhotometry.cgi? ra=280.8058788&dec=45.2077645&jdstart=2458231.891227&jdend=2458345.025359& email=xxxx@caltech.edu" Incaseyou’reinterested,thisexamplereturnsasubsetofmeasurementscoveringapreviouslydiscoveredTypeIaSuperluminousSupernova.4WhocansubmitarequestandhowwillIbenotified?Onlymeasurementsonepochalimagesforwhichusersareauthorizedtoaccessfromthearchive3(includinganyalready-publicly-releaseddata)willbereturned.Userswillbenotifiedofcompletedjobsviaemail.Theaddressusedistheoneyousuppliedonthewgetcommand-line(seeSection3).ThisemailwillcontainURLspointingtothelocationofyourforced-photometryproducts.YouwillbeabletodownloadyourproductsfromtheseURLsusingthesameUNAMEandPASSWDfromabove.Youwillreceiveaseparateemailforeachsubmittedrequest.

3Usingthearchiveportal:https://irsa.ipac.caltech.edu/Missions/ztf.html.Theforcedphotometryserviceusesthesamearchive-accessprivileges(basedonsurveyprogramID)thatwereinitiallyassignedwhenyouregisteredforanIRSAaccount.

4

5Rawoutputtable:whatdoesitmean?TheoutputisatableinASCIIformat.Thisconsistsofmetadataandforced-photometrymeasurementsforallobservationepochsthatyouwereauthorizedtoaccess.Anypublicly-releaseddata(fromthepublicsurvey)isincludedbydefault.Aheaderdescribingthetablecolumnsisprovided.Anexampleoutputtable(withthefirsttworowsofdata)isshownonpages5–6.Note:Thistabledoesnotrepresentastand-alonelightcurvethatyoucanimmediatelyplot.Itcontainsdifference-imageforcedphotometrymeasurementsontherequestedinputpositionandJDrangethatcoversallfields,CCD-quadrants,andfilters(g,r,ori)forwhichdifference-imagesexistinthearchive.Beforegeneratingalightcurve(Sections10and11),youwillneedtoanalyzethecontentsofthisfileandextracttherows(epochs)forthespecificfieldIDandfilterofinterest,includinganyqualitymetricsyouwishtofilteron.MultiplefieldIDsmaycoveryourtarget,andeachfieldID(withspecificCCD-quadranttherein)willhaveitsownreferenceimage,whichmayhavebeencreatedfromentirelydifferentinputobservationepochs.Youwillneedtotreateachfieldforagivenfilterseparately,toaccountforpossiblecontaminationfromatransient’sflux(seeSection8fordetails).Ignoringallthequalitymetricsyoumaywanttothreshold,thebareminimumforgeneratingaforcedPSF-fit-photometrydifferentiallightcurveforagivenZTFfieldandfilterarecolumns:field,filter,jd,zpdiff,forcediffimflux,andforcediffimfluxunc.

5

# ------------------------------------------------------------------ # Generated by forcedphotometry.pl, v1.5 on 2019-02-04 at 16:21:54 # # Requested input R.A. = 220.1058788 degrees # Requested input Dec. = 46.2077645 degrees # Requested JD start = 2458231.891227 days # Requested JD end = 2458345.025359 days # Authenticated program IDs = 0, 1, 2, 3 # # Column definitions: # index = sequential counter # field = ZTF Field identifier # ccdid = CCD identifier (1..16) # qid = Quadrant (CCD-amplifier) identifier (1..4) # filter = Filter string: ZTF_g, ZTF_r, ZTF_i # pid = processed image Operations DB identifier # infobitssci = processing summary/QA bits for sci image # airmass = Airmass value at start time of exposure where 1 => zenith # moonalt = Moon altitude above horizon at start time of exposure [deg] # moonillf = Moon illumination fraction at start time of exposure where 1 => full # moonra = R.A. of Moon at start time of exposure [deg] # moondec = Dec. of Moon at start time of exposure [deg] # sciinpseeing = Effective FWHM of sci image [pixels] # scibckgnd = Background level in sci image [DN] # scisigpix = Robust sigma per pixel in sci image [DN] # scimaglim = Magnitude limit in sci image [mag] # zpmaginpsci = Photometric zeropoint for sci image [mag] # zpmaginpsciunc = 1-sigma uncertainty in zpmaginpsci [mag] # zpmaginpscirms = RMS (deviation from average) in difference between instrumental mags and PS1 calibrators [mag] # clrcoeff = Linear color coefficient from calibration; for ZTF_g,r,i, PS1 color used is g-r, g-r, r-i respectively # clrcoeffunc = 1-sigma uncertainty in clrcoeff # ncalmatches = Number of PS1 calibrators used in initial calibration of sci image # exptime = Integration time for sci image [sec] # diffmaglim = Magnitude limit in difference image [mag] # zpdiff = Photometric zeropoint for difference image [mag] # programid = Program identifier [0=engineering; 1=public; 2=private; 3=Caltech] # obsdate = Observation date/time at start of exposure [UT] # jd = Julian Date at start of exposure [days] # scifilename = Archived sci image filename # diffilename = Archived difference image filename # rfid = Rerence image Operations DB identifier # refmaglim = Magnitude limit in reference image [mag] # refbckgnd = Background level in ref image [DN] # zpref = Photometric zeropoint for ref image [mag] # refcreated = Creation date/time of reference image [PT] # refjdstart = JD of earliest sci image used for ref image [days] # refjdend = JD of latest sci image used for ref image [days] # reffilename = Archived ref image filename # forcediffimflux = Forced difference image PSF-fit flux [DN] # forcediffimfluxunc = 1-sigma uncertainty in forcediffimflux [DN] # forcediffimsnr = Signal-to-noise ratio for forcediffimflux measurement # forcediffimchisq = Reduced chi-square in PSF-fit # forcediffimfluxap = Forced difference image aperture flux using a fixed 9-pixel diameter aperture [DN] # forcediffimfluxuncap = 1-sigma uncertainty in forcediffimfluxap [DN] # forcediffimsnrap = Signal-to-noise ratio for forcediffimfluxap measurement # aperturecorr = Aperture (curve-of-growth) correction factor that was applied to forcediffimfluxap measurement # dnearestrefsrc = Distance to nearest ref image source if found within 5 arcsec [arcsec] # nearestrefmag = Magnitude of nearest ref image source [mag] # nearestrefmagunc = 1-sigma uncertainty in nearestrefmag [mag] # nearestrefchi = Chi parameter for nearestrefmag estimate (ratio: RMS in PSF-fit residuals / expected RMS from priors)

6

# nearestrefsharp = Sharp parameter for nearestrefmag(~0 => PSF like; >>0 => extended; <<0 => pixel spike or hard edge) # procstatus = Per-epoch processing status codes (0 => no warnings); if non-zero, see accompanying log file and document link below (Section 7). # # Documentation: https://zwicky.tf/vgt # # ------------------------------------------------------------------ # Order of columns below: index, field, ccdid, qid, filter, pid, infobitssci, airmass, moonalt, moonillf, moonra, moondec, sciinpseeing, scibckgnd, scisigpix, scimaglim, zpmaginpsci, zpmaginpsciunc, zpmaginpscirms, clrcoeff, clrcoeffunc, ncalmatches, exptime, diffmaglim, zpdiff, programid, obsdate, jd, scifilename, diffilename, rfid, refmaglim, refbckgnd, refsigpix, zpref, refcreated, refjdstart, refjdend, reffilename, forcediffimflux, forcediffimfluxunc, forcediffimsnr, forcediffimchisq, forcediffimfluxap, forcediffimfluxuncap, forcediffimsnrap, aperturecorr, dnearestrefsrc, nearestrefmag, nearestrefmagunc, nearestrefchi, nearestrefsharp, procstatus # 0 765 3 1 ZTF_g 478442710915 0 1.107 -7.43321 0.671844 146.075 13.9642 2.3728 88.1144 11.971 20.69 25.383 5.9874e-06 0.0351354 -0.025068 1.77183e-05 1922 30 19.8779 26.325 1 2018-04-24T10:37:31.918 2458232.9427199 /ztf/archive/sci/2018/0424/442674/ztf_20180424442674_000765_zg_c03_o_q2_sciimg.fits /ztf/archive/sci/2018/0424/442674/ztf_20180424442674_000765_zg_c03_o_q2_scimrefdiffimg.fits.fz 765120109 23.05 87.4637 1.30273 26.325 2018-04-23T07:41:21.520467 2458204.976424 2458231.991227 /ztf/archive/ref/000/field000765/zg/ccd03/q2/ztf_000765_zg_c03_q2_refimg.fits 316.077406494178 100.3930057379015 2.57944323079164 0.90764381593827 106.035373149692 88.6726880094591 1.88945735460765 1.0697337179635 0.437751911135362 21.5740000724792 0.0820000022649765 0.874000012874603 -0.130999997258186 59 1 765 3 1 ZTF_r 478477960915 0 1.054 -16.6663 0.675564 146.643 13.8041 2.0241 87.6629 4.41207 21.75 26.2983 1.4891e-06 0.0174937 -0.0324827 4.43556e-06 1929 30 21.0069 26.325 1 2018-04-24T11:28:16.062 2458232.977963 /ztf/archive/sci/2018/0424/477905/ztf_20180424477905_000765_zr_c03_o_q2_sciimg.fits /ztf/archive/sci/2018/0424/477905/ztf_20180424477905_000765_zr_c03_o_q2_scimrefdiffimg.fits.fz 765120109 23.05 87.4278 1.29181 26.325 2018-04-23T07:41:21.520467 2458204.976424 2458231.991227 /ztf/archive/ref/000/field000765/zr/ccd03/q2/ztf_000765_zr_c03_q2_refimg.fits 112.545391010433 19.4421589168611 6.48233941317607 0.967813378063174 146.724395320634 33.2375251483508 4.4545105818558 1.06193141639954 0.437751911135362 21.5740000724792 0.0820000022649765 0.874000012874603 -0.130999997258186 56,59

…

7

6Advisoriesandcautionarynotes

(a) Forcedphotometryrequestswillbeprocessedataslowerratebetweenthehoursofsunsettosunrise(PacificTime).Thisisduetothelimitedresourcesavailableduringnightlypipelineoperations.

(b) Theoutputtablecontainsdifference-imageforcedphotometrymeasurementsontherequestedinputpositionandJDrangethatcoversallfields,CCD-quadrants,andfilters(g,r,ori)forwhichdifference-imagesexistinthearchive.

(c) Theforcedphotometryserviceonlyoperatesondifferenceimagesthatalreadyexistinthearchive.TheearliestobservationJDinyourreturnedtablereflectstheearliestdifferenceimagethatexists,forthegivenfield,CCD-quadrant,andfilter.Thisiscontingentonwhenthecorrespondingreferenceimagewascreated,andnotontheearliestscienceimagethatexists.Thereference-imagecreationtimestampisprovidedintheoutputtable:refcreated.Generatingdifferenceimagesontheflyiscomputationallyexpensive,althoughunderspecialcircumstances,wemaybeabletogeneratedifferenceimagesforolderexposures(pre-datingthereferenceimage)foragivenfield.

(d) IfarequestsuppliesaJD_STARTthatisearlierthantheofficialstartoftheZTFsciencesurvey(2018-03-17T00:00:00UTorJD=2458194.5),itisinternallyresettothelatter.

(e) WerecommendusingthePSF-fitphotometrymeasurementsforalllightcurves(forcediffimfluxandforcediffimfluxunc).Curve-of-growth-correctedaperturephotometrymeasurementsarealsoprovided(forcediffimfluxapandforcediffimfluxuncap).ThelatterarenoisierandservemoreasasanitycheckofthePSF-fitphotometry.

(f) Allcolumnsintheoutputtableprefixedwith“forced…”refertodifferentialfluxmeasurements.Thesearerelativetothefluxlevelinthearchivedreferenceimage.Weadviseexaminingthetimerangeoftheinputdatausedtogeneratetheinitialreferenceimage:refjdstartandrefjdendforpossiblecontaminationbytheactualtransientflux.Tobesure,wealsoadvisevisuallyexaminingthereferenceimage.Iftransientfluxispresent(andyouwillseethisinthetimeseriesofthehistoricaland/orfuturedifferentialfluxes),youwillneedtoperformabaselinecorrection(seeSection8).

(g) Ifyourtargetisavariablestarorsomeother(a)periodicsource,thereferenceimagewilllikelycontainatime-averagedsignalfromthesource.Inthiscase,youmaybeinterestedingeneratinganabsolutephotometriclightcurvebycombiningthedifferentialfluxeswiththemagnitudeofthenearestreference-imagecatalogsource:nearestrefmag,butfirstcheckingitsdistancefromyourtarget:dnearestrefsrc.SeeSection11fordetails.

(h) Bewareofbadpixelsimpactingtheindividualepochalmeasurements.Thiswillbeindicatedbyaprocstatuscontainingthevalue56.SeeSection7forcodedefinitions.

(i) AsignificantfractionoftheZTFdataisnon-photometric,thatis,wasacquiredthroughintermittentcloudcoverand/orwascontaminatedbythemoon.Thiswouldhaveimpactedthephotometriccalibration.Itisadvisedtotrendthecalibrationandnoisemetricsprovidedintheoutputtable:zpmaginpsci,zpmaginpscirms,and/orscisigpix.Moonposition(moonra,moondec)relativetoyourtarget,andilluminationfraction(moonillf)canalsobetrended.Outlierscanthenbeidentifiedinthesetrendsandomittedfromyourfinallightcurve.Wefindthescisigpixmetrictobethemostuseful.

(j) Thejdandobsdate(UT)timestampspertaintothestartofanexposure(nominally30s).(k) AllastrometrypertainstoequinoxJ2000andallarchivedproductswereastrometrically

calibratedwithrespecttoGaiaDR1.

8

7Exitandwarningcodes(theprocstatuscolumn)

Overallexitandwarningcodesfromexecutionofyourrequestareprovidedinthelogfilethataccompaniestheoutputtable.Processingstatuscodesarealsoprovidedonaper-epochbasis.Thisistheprocstatuscolumnintheoutputtable.Thesecodesaredefinedbelow.

- 0 => Successful execution - 56 => One or more epochs have photometry measurements that may be impacted by bad (including NaN'd) pixels - 57 => One or more epochs had no reference image catalog source falling with 5 arcsec - 58 => One or more epochs had a reference image PSF-catalog that does not exist in the archive - 59 => One or more epochs may have suspect photometric uncertainties due to early creation date of difference image in production - 60 => One or more epochs had upsampled diff-image PSF dimensions that were not odd integers - 61 => One or more epochs had diff-image cutouts that were off the image or too close to an edge - 62 => Requested start JD was before official survey start date [3/17/18] and was reset to 2018-03-17T00:00:00.0 UT - 63 => No records (epochs) returned by database query

- 64 => Catastrophic error (see log output) - 65 => Requested end JD is before official survey start date [3/17/18]

- 255 => Database connection or query execution error (see log output)

9

8BaselinecorrectionusingtherawfluxlightcurveofasuspecttransientForagivenfieldandfilterinyouroutputtable,itisrecommendedyouexamineaplotofdifferentialfluxforcediffimflux[DN]versustime[JD]todetermineifthereisanyresidualoffsetinthe“baseline”.Thebaselineherereferstotheleveldefinedbyeitherhistoricalorfutureepochs,wherethetransientsignalrelativetothereferenceimageisstationaryintime,relativetothebackgroundnoise.Forexample,ifthereferenceimageusedtogeneratethedifferenceimageswascontaminatedbythetransient’sflux,yourbaselinewillbe<0.Thereferenceimagemayalsohavebeenaffectedbysomeothersystematicinitsgenerationorcalibration,andthereforethebaselinecouldbe>0.Requestingaspecialtimerangeforregeneratingareference-imageforthepurposeofensuringazerobaselineinthedifferentialphotometryissuperfluous.Youwillalwaysneedtocheckandcorrectforapossiblenon-zerobaselinefortransients,duetopossiblesystematics,andhenceobtainthebestphotometricaccuracy.Thisisnotnecessaryfor(a)periodicvariablesthatmayfluctuateaboutaconstantlong-termflux-level.Thereasonisthatevenifthereferenceimagewasnotatrue(unbiased)time-averageofalltheinputepochs(atrandomlightcurvephases),anyresidualbaselinewillneedtoberetainedwhenconvertingfromdifferentialtoabsolutephotometryusingthereferenceimagesourcecatalogflux(seeSection11).Thebaselinecorrectionneedstouseasufficientnumberoffutureand/orhistoricalmeasurements.Wesuggest≥30epochs.Themorethebettersinceepochsmaybeaffectedbylowtransparency(cloudcover)and/ormoonglow,–seenote(i)inSection6.Itisassumedyouhavesomepriorknowledgeofthebehaviorofyourtransienttogobackorforwardenoughintimetoincludeasufficientnumberofepochsbeforesubmittingyourrequest(Section2).Figure1showsanexampleofatransient.Thefluxversustimeplotsingandrareonthetoprow.Ther-filterfluxplotontherightshowsafuturebaselineof~–90DN(dashedredline),asinferredfromthedifferentialfluxesatepochswithrelativeJD<60dayswherethetransientreachesquiescence.Atrimmedaverageormedianofthemeasurementsinthis“stationary”period,guidedbyvisualexaminationofthefluxlightcurveisrecommended.Followinganestimateofthebaselinelevel,youwillneedtosubtractthisfromallthedifferentialfluxmeasurements(forcediffimflux)beforeconvertingthemtocalibratedmagnitudesandupperlimits(seeSection10).

10

Figure1:toptobottomrow:(i)g&rlightcurvesinfluxspace;(ii)incalibratedmagnitudeswhereredtrianglesdenotesingle-epochupperlimits;(iii)alsoinmagnitudesbutcombiningthemeasurementsintheshadedwindowsshowninthefluxplots(blacksymbols).

11

9ValidatingphotometricuncertaintiesandpossiblerescalingDespitevalianteffortstoensurethe1-σuncertaintiesinthePSF-fitfluxes(forcediffimfluxunc)arecorrect,itisadvisedtocheckthattheseareplausible,atleastwithrespecttothephotometricrepeatabilityexpectedatagivenmagnitude,forthefieldandfilterbeingconsidered,i.e.,inthefrequentistsense.Themeasurementuncertaintiesarebasedonpropagatingasemi-empiricalmodelofthestatistical(random)noiseexpectedinthedetectorpixelsthroughthedifference-(andreference-)imagepipelines.Thismodeldoesnotaccountforpossiblesystematics,forexample,errorsinPSFestimation,photometriczero-pointandcolor-termcalibrations,flatfieldingerrors,astrometriccalibrations(whichdetermineultimatePSF-placement),and/oruser-errorinthesuppliedtargetposition.First,weadvisecheckingthedistributionofthePSF-fitreducedχ2values(forcediffimchisq)foragivenfieldandfilter,followinganyqualityfiltering(seenote(i)inSection6).Thiscanalsobeexaminedasafunctionofflux(forcediffimflux).Theaverage,<forcediffimchisq>,shouldbe≈1.Ifnot,weadvisemultiplyingtherawforcediffimfluxuncvaluesby√<forcediffimchisq>.Anyflux-dependentforcediffimchisqwillimplythe[e-/DN]gain-factorforestimatingthePoisson-noisecomponentisnotproperlytuned.Itcouldalsopointtoaflux-dependentPSF(!).Version1.0ofthesoftwaremayneedfurthertuningofthegain-factorsacrossallCCD-quadrants.Ensuring<forcediffimchisq>≈1onlymeansthatthepixeluncertaintiesareconsistent,inastatisticalsense,withthePSF-fitresidualsperepoch.Itwillnotcatchandcorrectforpossiblesystematicsacrossepochs(seebelow).Followingtheforcediffimchisqcheckandpossiblerescaling,youcanstophereorcontinuewithanotherchecktoaccountforpossibletemporalsystematics.ThiscanbedonebycomputingarobustRMS(e.g.,via0.5×[84th–16thpercentile])inthesignal-to-noise(S/N)ratios(forcediffimflux/forcediffimfluxuncvalues)ofthe“stationary-flux”epochsusedtoinferyourbaselinelevel(Section8).Theseratioswouldbecomputedfollowinganybaselinecorrectiontotheforcediffimfluxvaluesandanypriorscalingoftheforcediffimfluxunc.Iftheuncertaintiesareplausible,thisRMSshouldbe≈1.Ifnot,theforcediffimfluxuncvaluescanbemultipliedbythisRMS.ThiswillensuretheresultingdistributioninS/Nwillhavezero-meanandunitvariance.Bewareofepochswithbaddata(seenote(i)inSection6).Thisassumesthenoiseis≈stationaryduringthesehistorical/futureepochsandthattheysample,tofirstorder,thesameunderlyingbackgroundandconditionsasinepochswherethetransientwasdetected.It’salsoimportanttonotethatthischeckalonewillnotvalidatethePoisson-noisecomponentinepochswhenthetransienthashighsignal-to-noise.Afinalcoarsecheck(forvalidatinginparticularthePoisson-noise-dominatedregime)canbeperformedbycomparingagainstplotsofprior-calibratedphotometric-repeatability(stack-RMS)versusmagnitude4.Thiswillrequireyoutofirstconvertthefluxestocalibratedmagnitudes(seeSection10or11).Beware:thesepriorswillbefieldandfilterdependent.Afutureversionoftheforcedphotometrysoftwarewillperformaninternalsanitycheckagainstthesepriorswithautomaticrescalingoftheuncertainties.4Forexample,seeFigures9and10inhttps://zwicky.tf/3o9

12

10Generatingdifferential-photometrylightcurveswithnon-detectionsItisassumedyouhavecorrectedyourdifferentialfluxes(forcediffimflux)foranynon-zerobaseline(Section8)andvalidatedandpossiblyrescaledtheuncertainties(forcediffimfluxunc;Section9).Iftheuncertaintiesareover-orunderestimated,yoursignificancelevelsandhencederivedupperlimitswillbewrong.I.e.,“5-sigma”onlyhasmeaningif“sigma”iscorrect,thatis,itcapturesallnoisesourcesthatmayhavecorruptedthemeasurements.Thepseudo-codeforobtainingcalibratedmagnitudesisbelow.Thisusesthefollowingtwoparameters:SNT–thesignal-to-noisethresholdfordeclaringameasurementa“non-detection”soitcanbeassignedanupper-limit.SNU–theactualsignal-to-noiseratiotousewhencomputinga“SNU-sigma”upper-limit.IrecommendSNT=3andSNU=5.FordetailsonwhyIpickedthese,see:http://web.ipac.caltech.edu/staff/fmasci/home/mystats/UpperLimits_FM2011.pdf

if( (forcediffimflux / forcediffimfluxunc) > SNT ) # we have a “confident” detection, compute and plot mag with error bar: mag = zpdiff – 2.5*log10[forcediffimflux] σmag = 1.0857* forcediffimfluxunc / forcediffimflux

else # compute flux upper limit and plot as arrow: mag = zpdiff – 2.5*log10[SNU*forcediffimfluxunc]

ExamplelightcurvesingandrfollowingthisconstructionareshowninthesecondrowofFigure1.11Generatingabsolute-photometrylightcurvesforvariablesourcesTransientsarecommonlyassociatedwithsuddeneventswheresignalsuddenlyappearsoutofthenoiseandthenfades.Thegoalistoanalyzetheenergyreleasedduringtheexplosivephase,inexcessofanypre-existingsignal.Forcontinuous(a)periodicvariablesortransientsassociatedwithpre-quiescentsourceswithadetectablesignal,onemaywanttocharacterizetheabsolute(ortotal)signalversustime.Here,youwillneedanestimateofthesourcefluxfromthereference-imagethatwasusedtogeneratethedifferenceimageswhereforcedphotometrywasperformed.Photometryandqualitymetricsforthenearestreferenceimagesourcearegivenbycolumnsprefixedby“nearestref…”intheoutputtable.ThesearefromthearchivedPSF-fitreferenceimagecatalog.Youwillneedtothresholddnearestrefsrctoensureareferencesourceexistsclosetoyourtargetposition.Werecommenddnearestrefsrc<~1arcsec.ThenearestrefchiandnearestrefsharpmetricscanbeusedtocheckifthereferencesourceisextendedorPSF-like.Ifsignificantlyextended,thePSF-fitphotometry(nearestrefmag)willbeerroneous.Youwillwanttoensureitisapointsource.

13

Belowarethestepsforobtaininganabsolutephotometriclightcurve(modulatedbyvariationsinferredfromdifferentialphotometry)incalibratedmagnitudes.(a) Computethereferencesourcefluxanditsuncertaintyonthesamecalibration(zero-

point)asthedifferentialfluxateachepoch:nearestrefflux=10**[0.4*(zpdiff–nearestrefmag)];nearestreffluxunc=nearestrefmagunc*nearestrefflux/1.0857,wherezpdiff,nearestrefmag,andnearestrefmaguncaregivenintheoutputtable.

(b) Computethetotalflux,itsuncertainty,andsignal-to-noiseratioateachepoch:Fluxtot=forcediffimflux+nearestrefflux;Fluxunctot=√(forcediffimfluxunc2–nearestreffluxunc2);SNRtot=Fluxtot/Fluxunctot.Thenegativesigninthesquare-rootfollowsfromthe(implicit)assumptionthatthereferencefluxwasmeasuredfromthesamereferenceimageusedtogeneratethe“science–reference”differentialflux.Hence,thenoiseisanti-correlated.Ifthisisnottrueoriftheargumentinthesquare-rootis≤0forwhateverreason,wesuggestaddingthevariancesinstead.Thiswillbeaconservativeestimate.

(c) Usingtheoutputsfrom(b)andtheSNT,SNUparametersdefinedinSection10,here’sthepseudo-codeforgeneratinganabsolute-photometrylightcurve:

if( SNRtot > SNT )

# we have a “confident” detection, compute and plot mag with error bar: mag = zpdiff – 2.5*log10[Fluxtot] σmag = 1.0857 / SNRtot

else # compute flux upper limit and plot as arrow: mag = zpdiff – 2.5*log10[SNU*Fluxunctot]

Thislogicreducestothedifferential-lightcurvegenerationlogicinSection10whennearestrefflux=0andFluxunctot=forcediffimfluxunc.

TheaboveconstructiontocomputecalibratedmagnitudesassumesyoursourcehaszerocolorinthePan-STARRS1photometricsystem(gPS1–rPS1=0forgandrfluxmeasurements;rPS1–iPS1=0forifluxmeasurements).Ifyouhavepriorinformationonthecolororspectrumofyoursourceinthesebandpassesacrossepochs,themagnitudecomputationsabovecanincludetheadditiveterm:+clrcoeff*(gPS1–rPS1)forgandrmeasurements,or+clrcoeff*(rPS1–iPS1)fori-filtermeasurements,whereclrcoeffisintheoutputtable.

14

12Goingdeeper:combiningsingle-epochmeasurementsIfyou’refeelingambitiousandwanttomakeyourlightcurvemeasurementsmorestatisticallysignificantand/orwithtighterupper-limits,youcanattempttocombinethefluxmeasurementswithincarefullyselectedtime-windowsusingsomeoptimalmethod.Thisprocedureassumesyoursingle-epochdifferentialfluxeshavebeencorrectedforanynon-zerobaseline(inthecaseoftransients;Section8)anduncertaintieshavebeenvalidatedandrescaledifnecessary(Section9).Onemethodistoassumetheunderlyingsourcesignalisstationarywithinatime-windowandcollapsethesingle-epochfluxes(forcediffimfluxi)thereinusinganinverse-varianceweightedaverage:

Flux =wi forcediffimfluxi

i∑

wii∑

where wi =1

forcediffimfluxunci2

withuncertainty:

Fluxunc = wii∑⎡

⎣⎢

⎤

⎦⎥

−1/2

.

Thelastlineiseffectively≈forceddiffimfluxunc/√nifoneassumesthenwindowedsingle-epochfluxmeasurementshaveapproximatelythesameuncertaintyequaltosomeconstantvalueorRMS.Therefore,theimprovementinsignal-to-noiseassuminguncorrelatedinputsisnomorethanafactorof≈√n.Theimportantassumptionhereisthattheunderlyingsourcesignalisconstantwithinthewindow.Itmayappearconstantwithinmeasurementerror,butwhenmanymeasurementsareavailable,youcanattempttobintheminvariouswaystoteaseoutpossiblehiddentrendsinthesignal.Amoving(weighted)averagemayalsowork,usingaproperlytunedwindowsize.There’salsoalargecollectionofmethodsonlocal-polynomialregressionfitting.Youcanalsoattempttocollapsethemeasurementswithinwindowsbyfittingapriormodeloffluxversustime,i.e.,ifyouhaveprior(orcontextual)knowledgefromotherhighersignal-to-noisedata.Theadvantagesofcombiningmeasurementsinsource-spaceacrossepochsasopposedtoco-addingentireimagesintime-orderedslicesare:(i)speedand(ii)greaterflexibilitywhencombiningthem.Havingcomputedthewindow-collapsedfluxes,youcanconvertthemtocalibratedmagnitudeswithupper-limitsusingtheformalisminSection10(fortransients)orSection11(forabsolutephotometryonvariables).ThetoprowinFigure1showsexamplewindows(shadedgreen)wherethesingle-epochfluxmeasurementsarecombined.Thesewerecombinedusingaweightedaverage(seeabove)andthecalibratedphotometrywithupperlimitsareshowninthebottomrow.