How to exclude values from a polynomial fit?

Multi tool use
I fit a polynomial to my data, as shown in the figure:
Using the script:
from scipy.optimize import curve_fit
import scipy.stats
from scipy import asarray as ar,exp
xdata = xvalues
ydata = yvalues
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
axes.plot(xdata, ydata, '-')
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
axes.plot(xModel, yModel)
I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.
python-3.x scipy polynomials
add a comment |
I fit a polynomial to my data, as shown in the figure:
Using the script:
from scipy.optimize import curve_fit
import scipy.stats
from scipy import asarray as ar,exp
xdata = xvalues
ydata = yvalues
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
axes.plot(xdata, ydata, '-')
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
axes.plot(xModel, yModel)
I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.
python-3.x scipy polynomials
add a comment |
I fit a polynomial to my data, as shown in the figure:
Using the script:
from scipy.optimize import curve_fit
import scipy.stats
from scipy import asarray as ar,exp
xdata = xvalues
ydata = yvalues
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
axes.plot(xdata, ydata, '-')
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
axes.plot(xModel, yModel)
I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.
python-3.x scipy polynomials
I fit a polynomial to my data, as shown in the figure:
Using the script:
from scipy.optimize import curve_fit
import scipy.stats
from scipy import asarray as ar,exp
xdata = xvalues
ydata = yvalues
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
axes.plot(xdata, ydata, '-')
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
axes.plot(xModel, yModel)
I want to exclude the region from 3.4 to 3.55 um. How can I do that in my script? Also I have NaNs that I am trying to get rid of in the original .fits file. Help would be valued.
python-3.x scipy polynomials
python-3.x scipy polynomials
asked Nov 13 '18 at 8:56
D.KimD.Kim
435
435
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You can mask the values within your exclustion region and apply this mask to your fit function later
# Using random data here, since you haven't provided sample data
xdata = numpy.arange(3,4,0.01)
ydata = 2* numpy.random.rand(len(xdata)) + xdata
# Create mask (boolean array) of values outside of your exclusion region
mask = (xdata < 3.4) | (xdata > 3.55)
# Do the fit on all data (for comparison)
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
# Do the fit on the masked data (i.e. only that data, where mask == True)
fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
yModel1 = numpy.polyval(fittedParameters1, xModel1)
# Plot stuff
axes.plot(xdata, ydata, '-')
axes.plot(xModel, yModel) # orange
axes.plot(xModel1, yModel1) # green
gives
The green curve is now the fit with 3.4 < xdata 3.55
excluded. The orange curve is the fitout without an exclusion (for comparison)
If you want to exclude also possible nans in your xdata
you can enhance the mask
by the numpy.isnan()
function like
# Create mask (boolean array) of values outside of your exclusion AND which ar not nan
xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53277172%2fhow-to-exclude-values-from-a-polynomial-fit%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can mask the values within your exclustion region and apply this mask to your fit function later
# Using random data here, since you haven't provided sample data
xdata = numpy.arange(3,4,0.01)
ydata = 2* numpy.random.rand(len(xdata)) + xdata
# Create mask (boolean array) of values outside of your exclusion region
mask = (xdata < 3.4) | (xdata > 3.55)
# Do the fit on all data (for comparison)
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
# Do the fit on the masked data (i.e. only that data, where mask == True)
fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
yModel1 = numpy.polyval(fittedParameters1, xModel1)
# Plot stuff
axes.plot(xdata, ydata, '-')
axes.plot(xModel, yModel) # orange
axes.plot(xModel1, yModel1) # green
gives
The green curve is now the fit with 3.4 < xdata 3.55
excluded. The orange curve is the fitout without an exclusion (for comparison)
If you want to exclude also possible nans in your xdata
you can enhance the mask
by the numpy.isnan()
function like
# Create mask (boolean array) of values outside of your exclusion AND which ar not nan
xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
add a comment |
You can mask the values within your exclustion region and apply this mask to your fit function later
# Using random data here, since you haven't provided sample data
xdata = numpy.arange(3,4,0.01)
ydata = 2* numpy.random.rand(len(xdata)) + xdata
# Create mask (boolean array) of values outside of your exclusion region
mask = (xdata < 3.4) | (xdata > 3.55)
# Do the fit on all data (for comparison)
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
# Do the fit on the masked data (i.e. only that data, where mask == True)
fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
yModel1 = numpy.polyval(fittedParameters1, xModel1)
# Plot stuff
axes.plot(xdata, ydata, '-')
axes.plot(xModel, yModel) # orange
axes.plot(xModel1, yModel1) # green
gives
The green curve is now the fit with 3.4 < xdata 3.55
excluded. The orange curve is the fitout without an exclusion (for comparison)
If you want to exclude also possible nans in your xdata
you can enhance the mask
by the numpy.isnan()
function like
# Create mask (boolean array) of values outside of your exclusion AND which ar not nan
xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
add a comment |
You can mask the values within your exclustion region and apply this mask to your fit function later
# Using random data here, since you haven't provided sample data
xdata = numpy.arange(3,4,0.01)
ydata = 2* numpy.random.rand(len(xdata)) + xdata
# Create mask (boolean array) of values outside of your exclusion region
mask = (xdata < 3.4) | (xdata > 3.55)
# Do the fit on all data (for comparison)
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
# Do the fit on the masked data (i.e. only that data, where mask == True)
fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
yModel1 = numpy.polyval(fittedParameters1, xModel1)
# Plot stuff
axes.plot(xdata, ydata, '-')
axes.plot(xModel, yModel) # orange
axes.plot(xModel1, yModel1) # green
gives
The green curve is now the fit with 3.4 < xdata 3.55
excluded. The orange curve is the fitout without an exclusion (for comparison)
If you want to exclude also possible nans in your xdata
you can enhance the mask
by the numpy.isnan()
function like
# Create mask (boolean array) of values outside of your exclusion AND which ar not nan
xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)
You can mask the values within your exclustion region and apply this mask to your fit function later
# Using random data here, since you haven't provided sample data
xdata = numpy.arange(3,4,0.01)
ydata = 2* numpy.random.rand(len(xdata)) + xdata
# Create mask (boolean array) of values outside of your exclusion region
mask = (xdata < 3.4) | (xdata > 3.55)
# Do the fit on all data (for comparison)
fittedParameters = numpy.polyfit(xdata, ydata + .00001005 , 3)
modelPredictions = numpy.polyval(fittedParameters, xdata)
xModel = numpy.linspace(min(xdata), max(xdata))
yModel = numpy.polyval(fittedParameters, xModel)
# Do the fit on the masked data (i.e. only that data, where mask == True)
fittedParameters1 = numpy.polyfit(xdata[mask], ydata[mask] + .00001005 , 3)
modelPredictions1 = numpy.polyval(fittedParameters1, xdata[mask])
xModel1 = numpy.linspace(min(xdata[mask]), max(xdata[mask]))
yModel1 = numpy.polyval(fittedParameters1, xModel1)
# Plot stuff
axes.plot(xdata, ydata, '-')
axes.plot(xModel, yModel) # orange
axes.plot(xModel1, yModel1) # green
gives
The green curve is now the fit with 3.4 < xdata 3.55
excluded. The orange curve is the fitout without an exclusion (for comparison)
If you want to exclude also possible nans in your xdata
you can enhance the mask
by the numpy.isnan()
function like
# Create mask (boolean array) of values outside of your exclusion AND which ar not nan
xdata < 3.4) | (xdata > 3.55) & ~numpy.isnan(xdata)
answered Nov 13 '18 at 9:54
gehbiszumeisgehbiszumeis
859418
859418
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
add a comment |
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
That worked thanks!
– D.Kim
Nov 13 '18 at 10:33
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
When I try to divide the original data (e.g., ydata) by the curve with the masked data (e.g., ymodel1), now the dimensions are different, and I cannot divide the two. How to go around this?
– D.Kim
Nov 13 '18 at 13:12
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53277172%2fhow-to-exclude-values-from-a-polynomial-fit%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
68vb,rEvaiglzEgBbZ hoVWRpowdigUGh7UTt8oasEeu,VlGjppahZy7FjKjHZtchnpABHG6ieKsURnh es hV9sn JL d7