How to exclude values from a polynomial fit?










2














I fit a polynomial to my data, as shown in the figure: enter image description here



Using the script:



from scipy.optimize import curve_fit
import scipy.stats
from scipy import asarray as ar,exp

xdata = xvalues
ydata = yvalues

fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)

axes.plot(xdata, ydata, '-')
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)

axes.plot(xModel, yModel)


I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.










share|improve this question


























    2














    I fit a polynomial to my data, as shown in the figure: enter image description here



    Using the script:



    from scipy.optimize import curve_fit
    import scipy.stats
    from scipy import asarray as ar,exp

    xdata = xvalues
    ydata = yvalues

    fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
    modelPredictions = numpy.polyval(fittedParameters, xdata)

    axes.plot(xdata, ydata, '-')
    xModel = numpy.linspace(min(xdata), max(xdata))
    yModel = numpy.polyval(fittedParameters, xModel)

    axes.plot(xModel, yModel)


    I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.










    share|improve this question
























      2












      2








      2







      I fit a polynomial to my data, as shown in the figure: enter image description here



      Using the script:



      from scipy.optimize import curve_fit
      import scipy.stats
      from scipy import asarray as ar,exp

      xdata = xvalues
      ydata = yvalues

      fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
      modelPredictions = numpy.polyval(fittedParameters, xdata)

      axes.plot(xdata, ydata, '-')
      xModel = numpy.linspace(min(xdata), max(xdata))
      yModel = numpy.polyval(fittedParameters, xModel)

      axes.plot(xModel, yModel)


      I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.










      share|improve this question













      I fit a polynomial to my data, as shown in the figure: enter image description here



      Using the script:



      from scipy.optimize import curve_fit
      import scipy.stats
      from scipy import asarray as ar,exp

      xdata = xvalues
      ydata = yvalues

      fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
      modelPredictions = numpy.polyval(fittedParameters, xdata)

      axes.plot(xdata, ydata, '-')
      xModel = numpy.linspace(min(xdata), max(xdata))
      yModel = numpy.polyval(fittedParameters, xModel)

      axes.plot(xModel, yModel)


      I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.







      python-3.x scipy polynomials






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 13 '18 at 8:56









      D.KimD.Kim

      435




      435






















          1 Answer
          1






          active

          oldest

          votes


















          1














          You can mask the values within your exclustion region and apply this mask to your fit function later



          # Using random data here, since you haven't provided sample data
          xdata = numpy.arange(3,4,0.01)
          ydata = 2* numpy.random.rand(len(xdata)) + xdata

          # Create mask (boolean array) of values outside of your exclusion region
          mask = (xdata < 3.4) | (xdata > 3.55)

          # Do the fit on all data (for comparison)
          fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
          modelPredictions = numpy.polyval(fittedParameters, xdata)
          xModel = numpy.linspace(min(xdata), max(xdata))
          yModel = numpy.polyval(fittedParameters, xModel)

          # Do the fit on the masked data (i.e. only that data, where mask == True)
          fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
          modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
          xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
          yModel1 = numpy.polyval(fittedParameters1, xModel1)

          # Plot stuff
          axes.plot(xdata, ydata, '-')
          axes.plot(xModel, yModel) # orange
          axes.plot(xModel1, yModel1) # green


          gives



          enter image description here



          The green curve is now the fit with 3.4 < xdata 3.55 excluded. The orange curve is the fitout without an exclusion (for comparison)



          If you want to exclude also possible nans in your xdata you can enhance the mask by the numpy.isnan() function like



          # Create mask (boolean array) of values outside of your exclusion AND which ar not nan
          xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)





          share|improve this answer




















          • That worked thanks!
            – D.Kim
            Nov 13 '18 at 10:33










          • When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
            – D.Kim
            Nov 13 '18 at 13:12










          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53277172%2fhow-to-exclude-values-from-a-polynomial-fit%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          You can mask the values within your exclustion region and apply this mask to your fit function later



          # Using random data here, since you haven't provided sample data
          xdata = numpy.arange(3,4,0.01)
          ydata = 2* numpy.random.rand(len(xdata)) + xdata

          # Create mask (boolean array) of values outside of your exclusion region
          mask = (xdata < 3.4) | (xdata > 3.55)

          # Do the fit on all data (for comparison)
          fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
          modelPredictions = numpy.polyval(fittedParameters, xdata)
          xModel = numpy.linspace(min(xdata), max(xdata))
          yModel = numpy.polyval(fittedParameters, xModel)

          # Do the fit on the masked data (i.e. only that data, where mask == True)
          fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
          modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
          xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
          yModel1 = numpy.polyval(fittedParameters1, xModel1)

          # Plot stuff
          axes.plot(xdata, ydata, '-')
          axes.plot(xModel, yModel) # orange
          axes.plot(xModel1, yModel1) # green


          gives



          enter image description here



          The green curve is now the fit with 3.4 < xdata 3.55 excluded. The orange curve is the fitout without an exclusion (for comparison)



          If you want to exclude also possible nans in your xdata you can enhance the mask by the numpy.isnan() function like



          # Create mask (boolean array) of values outside of your exclusion AND which ar not nan
          xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)





          share|improve this answer




















          • That worked thanks!
            – D.Kim
            Nov 13 '18 at 10:33










          • When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
            – D.Kim
            Nov 13 '18 at 13:12















          1














          You can mask the values within your exclustion region and apply this mask to your fit function later



          # Using random data here, since you haven't provided sample data
          xdata = numpy.arange(3,4,0.01)
          ydata = 2* numpy.random.rand(len(xdata)) + xdata

          # Create mask (boolean array) of values outside of your exclusion region
          mask = (xdata < 3.4) | (xdata > 3.55)

          # Do the fit on all data (for comparison)
          fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
          modelPredictions = numpy.polyval(fittedParameters, xdata)
          xModel = numpy.linspace(min(xdata), max(xdata))
          yModel = numpy.polyval(fittedParameters, xModel)

          # Do the fit on the masked data (i.e. only that data, where mask == True)
          fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
          modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
          xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
          yModel1 = numpy.polyval(fittedParameters1, xModel1)

          # Plot stuff
          axes.plot(xdata, ydata, '-')
          axes.plot(xModel, yModel) # orange
          axes.plot(xModel1, yModel1) # green


          gives



          enter image description here



          The green curve is now the fit with 3.4 < xdata 3.55 excluded. The orange curve is the fitout without an exclusion (for comparison)



          If you want to exclude also possible nans in your xdata you can enhance the mask by the numpy.isnan() function like



          # Create mask (boolean array) of values outside of your exclusion AND which ar not nan
          xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)





          share|improve this answer




















          • That worked thanks!
            – D.Kim
            Nov 13 '18 at 10:33










          • When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
            – D.Kim
            Nov 13 '18 at 13:12













          1












          1








          1






          You can mask the values within your exclustion region and apply this mask to your fit function later



          # Using random data here, since you haven't provided sample data
          xdata = numpy.arange(3,4,0.01)
          ydata = 2* numpy.random.rand(len(xdata)) + xdata

          # Create mask (boolean array) of values outside of your exclusion region
          mask = (xdata < 3.4) | (xdata > 3.55)

          # Do the fit on all data (for comparison)
          fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
          modelPredictions = numpy.polyval(fittedParameters, xdata)
          xModel = numpy.linspace(min(xdata), max(xdata))
          yModel = numpy.polyval(fittedParameters, xModel)

          # Do the fit on the masked data (i.e. only that data, where mask == True)
          fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
          modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
          xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
          yModel1 = numpy.polyval(fittedParameters1, xModel1)

          # Plot stuff
          axes.plot(xdata, ydata, '-')
          axes.plot(xModel, yModel) # orange
          axes.plot(xModel1, yModel1) # green


          gives



          enter image description here



          The green curve is now the fit with 3.4 < xdata 3.55 excluded. The orange curve is the fitout without an exclusion (for comparison)



          If you want to exclude also possible nans in your xdata you can enhance the mask by the numpy.isnan() function like



          # Create mask (boolean array) of values outside of your exclusion AND which ar not nan
          xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)





          share|improve this answer












          You can mask the values within your exclustion region and apply this mask to your fit function later



          # Using random data here, since you haven't provided sample data
          xdata = numpy.arange(3,4,0.01)
          ydata = 2* numpy.random.rand(len(xdata)) + xdata

          # Create mask (boolean array) of values outside of your exclusion region
          mask = (xdata < 3.4) | (xdata > 3.55)

          # Do the fit on all data (for comparison)
          fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
          modelPredictions = numpy.polyval(fittedParameters, xdata)
          xModel = numpy.linspace(min(xdata), max(xdata))
          yModel = numpy.polyval(fittedParameters, xModel)

          # Do the fit on the masked data (i.e. only that data, where mask == True)
          fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
          modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
          xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
          yModel1 = numpy.polyval(fittedParameters1, xModel1)

          # Plot stuff
          axes.plot(xdata, ydata, '-')
          axes.plot(xModel, yModel) # orange
          axes.plot(xModel1, yModel1) # green


          gives



          enter image description here



          The green curve is now the fit with 3.4 < xdata 3.55 excluded. The orange curve is the fitout without an exclusion (for comparison)



          If you want to exclude also possible nans in your xdata you can enhance the mask by the numpy.isnan() function like



          # Create mask (boolean array) of values outside of your exclusion AND which ar not nan
          xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 13 '18 at 9:54









          gehbiszumeisgehbiszumeis

          859418




          859418











          • That worked thanks!
            – D.Kim
            Nov 13 '18 at 10:33










          • When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
            – D.Kim
            Nov 13 '18 at 13:12
















          • That worked thanks!
            – D.Kim
            Nov 13 '18 at 10:33










          • When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
            – D.Kim
            Nov 13 '18 at 13:12















          That worked thanks!
          – D.Kim
          Nov 13 '18 at 10:33




          That worked thanks!
          – D.Kim
          Nov 13 '18 at 10:33












          When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
          – D.Kim
          Nov 13 '18 at 13:12




          When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
          – D.Kim
          Nov 13 '18 at 13:12

















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53277172%2fhow-to-exclude-values-from-a-polynomial-fit%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Top Tejano songwriter Luis Silva dead of heart attack at 64

          ReactJS Fetched API data displays live - need Data displayed static

          Evgeni Malkin