Slice NetCdf Dataset
I am looking to take a subset of the netcdf data set bounded by lat/lon coordinates.
<xarray.Dataset>
Dimensions: (ICcheckNameLen: 72, ICcheckNum: 55, QCcheckNameLen: 60, QCcheckNum: 10, maxAutoStaLen: 6, maxLocationLen: 24, maxMETARLen: 256, maxRepLen: 6, maxSkyCover: 6, maxSkyLen: 8, maxStaNamLen: 5, maxStaticIds: 10000, maxWeatherLen: 25, nInventoryBins: 24, recNum: 8329, totalIdLen: 6)
Dimensions without coordinates: ICcheckNameLen, ICcheckNum, QCcheckNameLen, QCcheckNum, maxAutoStaLen, maxLocationLen, maxMETARLen, maxRepLen, maxSkyCover, maxSkyLen, maxStaNamLen, maxStaticIds, maxWeatherLen, nInventoryBins, recNum, totalIdLen
Data variables:
nStaticIds int32 ...
staticIds (maxStaticIds, totalIdLen) |S1 ...
lastRecord (maxStaticIds) int32 ...
invTime (recNum) int32 ...
prevRecord (recNum) int32 ...
inventory (maxStaticIds) int32 ...
globalInventory int32 ...
firstOverflow int32 ...
isOverflow (recNum) int32 ...
firstInBin (nInventoryBins) int32 ...
lastInBin (nInventoryBins) int32 ...
secondsStage1_2 (recNum) int32 ...
secondsStage3 (recNum) int32 ...
wmoId (recNum) int32 ...
stationName (recNum, maxStaNamLen) |S1 ...
locationName (recNum, maxLocationLen) |S1 ...
QCT (QCcheckNum, QCcheckNameLen) |S1 ...
ICT (ICcheckNum, ICcheckNameLen) |S1 ...
latitude (recNum) float32 ...
longitude (recNum) float32 ...
elevation (recNum) float32 ...
I have tried multiple methods based on Help1 and Help2 to setup the boundaries which should be between latitude[20,53] and longitude[-131,-62]. The dataset can be accessed at NetCDF Data.
When I use the below, it says, "ValueError: dimensions or multi-index levels ['latitude', 'longitude'] do not exist"
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc',
decode_cf=False)
print(ds)
lat_bnds, lon_bnds = [20, 53], [-131, -62]
ds.sel(latitude=slice(*lat_bnds), longitude=slice(*lon_bnds))
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
When I try the below, it works through the data, but does not remove any data.
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc', decode_cf=True)
ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
Any ideas?
python slice netcdf python-xarray
add a comment |
I am looking to take a subset of the netcdf data set bounded by lat/lon coordinates.
<xarray.Dataset>
Dimensions: (ICcheckNameLen: 72, ICcheckNum: 55, QCcheckNameLen: 60, QCcheckNum: 10, maxAutoStaLen: 6, maxLocationLen: 24, maxMETARLen: 256, maxRepLen: 6, maxSkyCover: 6, maxSkyLen: 8, maxStaNamLen: 5, maxStaticIds: 10000, maxWeatherLen: 25, nInventoryBins: 24, recNum: 8329, totalIdLen: 6)
Dimensions without coordinates: ICcheckNameLen, ICcheckNum, QCcheckNameLen, QCcheckNum, maxAutoStaLen, maxLocationLen, maxMETARLen, maxRepLen, maxSkyCover, maxSkyLen, maxStaNamLen, maxStaticIds, maxWeatherLen, nInventoryBins, recNum, totalIdLen
Data variables:
nStaticIds int32 ...
staticIds (maxStaticIds, totalIdLen) |S1 ...
lastRecord (maxStaticIds) int32 ...
invTime (recNum) int32 ...
prevRecord (recNum) int32 ...
inventory (maxStaticIds) int32 ...
globalInventory int32 ...
firstOverflow int32 ...
isOverflow (recNum) int32 ...
firstInBin (nInventoryBins) int32 ...
lastInBin (nInventoryBins) int32 ...
secondsStage1_2 (recNum) int32 ...
secondsStage3 (recNum) int32 ...
wmoId (recNum) int32 ...
stationName (recNum, maxStaNamLen) |S1 ...
locationName (recNum, maxLocationLen) |S1 ...
QCT (QCcheckNum, QCcheckNameLen) |S1 ...
ICT (ICcheckNum, ICcheckNameLen) |S1 ...
latitude (recNum) float32 ...
longitude (recNum) float32 ...
elevation (recNum) float32 ...
I have tried multiple methods based on Help1 and Help2 to setup the boundaries which should be between latitude[20,53] and longitude[-131,-62]. The dataset can be accessed at NetCDF Data.
When I use the below, it says, "ValueError: dimensions or multi-index levels ['latitude', 'longitude'] do not exist"
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc',
decode_cf=False)
print(ds)
lat_bnds, lon_bnds = [20, 53], [-131, -62]
ds.sel(latitude=slice(*lat_bnds), longitude=slice(*lon_bnds))
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
When I try the below, it works through the data, but does not remove any data.
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc', decode_cf=True)
ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
Any ideas?
python slice netcdf python-xarray
Your second approach looks like the right way to solve this, for cases wherelatitudeandlongitudeare not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?
– shoyer
Nov 15 '18 at 21:20
I changed the code tods.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.
– WxJack
Nov 15 '18 at 21:49
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59
add a comment |
I am looking to take a subset of the netcdf data set bounded by lat/lon coordinates.
<xarray.Dataset>
Dimensions: (ICcheckNameLen: 72, ICcheckNum: 55, QCcheckNameLen: 60, QCcheckNum: 10, maxAutoStaLen: 6, maxLocationLen: 24, maxMETARLen: 256, maxRepLen: 6, maxSkyCover: 6, maxSkyLen: 8, maxStaNamLen: 5, maxStaticIds: 10000, maxWeatherLen: 25, nInventoryBins: 24, recNum: 8329, totalIdLen: 6)
Dimensions without coordinates: ICcheckNameLen, ICcheckNum, QCcheckNameLen, QCcheckNum, maxAutoStaLen, maxLocationLen, maxMETARLen, maxRepLen, maxSkyCover, maxSkyLen, maxStaNamLen, maxStaticIds, maxWeatherLen, nInventoryBins, recNum, totalIdLen
Data variables:
nStaticIds int32 ...
staticIds (maxStaticIds, totalIdLen) |S1 ...
lastRecord (maxStaticIds) int32 ...
invTime (recNum) int32 ...
prevRecord (recNum) int32 ...
inventory (maxStaticIds) int32 ...
globalInventory int32 ...
firstOverflow int32 ...
isOverflow (recNum) int32 ...
firstInBin (nInventoryBins) int32 ...
lastInBin (nInventoryBins) int32 ...
secondsStage1_2 (recNum) int32 ...
secondsStage3 (recNum) int32 ...
wmoId (recNum) int32 ...
stationName (recNum, maxStaNamLen) |S1 ...
locationName (recNum, maxLocationLen) |S1 ...
QCT (QCcheckNum, QCcheckNameLen) |S1 ...
ICT (ICcheckNum, ICcheckNameLen) |S1 ...
latitude (recNum) float32 ...
longitude (recNum) float32 ...
elevation (recNum) float32 ...
I have tried multiple methods based on Help1 and Help2 to setup the boundaries which should be between latitude[20,53] and longitude[-131,-62]. The dataset can be accessed at NetCDF Data.
When I use the below, it says, "ValueError: dimensions or multi-index levels ['latitude', 'longitude'] do not exist"
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc',
decode_cf=False)
print(ds)
lat_bnds, lon_bnds = [20, 53], [-131, -62]
ds.sel(latitude=slice(*lat_bnds), longitude=slice(*lon_bnds))
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
When I try the below, it works through the data, but does not remove any data.
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc', decode_cf=True)
ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
Any ideas?
python slice netcdf python-xarray
I am looking to take a subset of the netcdf data set bounded by lat/lon coordinates.
<xarray.Dataset>
Dimensions: (ICcheckNameLen: 72, ICcheckNum: 55, QCcheckNameLen: 60, QCcheckNum: 10, maxAutoStaLen: 6, maxLocationLen: 24, maxMETARLen: 256, maxRepLen: 6, maxSkyCover: 6, maxSkyLen: 8, maxStaNamLen: 5, maxStaticIds: 10000, maxWeatherLen: 25, nInventoryBins: 24, recNum: 8329, totalIdLen: 6)
Dimensions without coordinates: ICcheckNameLen, ICcheckNum, QCcheckNameLen, QCcheckNum, maxAutoStaLen, maxLocationLen, maxMETARLen, maxRepLen, maxSkyCover, maxSkyLen, maxStaNamLen, maxStaticIds, maxWeatherLen, nInventoryBins, recNum, totalIdLen
Data variables:
nStaticIds int32 ...
staticIds (maxStaticIds, totalIdLen) |S1 ...
lastRecord (maxStaticIds) int32 ...
invTime (recNum) int32 ...
prevRecord (recNum) int32 ...
inventory (maxStaticIds) int32 ...
globalInventory int32 ...
firstOverflow int32 ...
isOverflow (recNum) int32 ...
firstInBin (nInventoryBins) int32 ...
lastInBin (nInventoryBins) int32 ...
secondsStage1_2 (recNum) int32 ...
secondsStage3 (recNum) int32 ...
wmoId (recNum) int32 ...
stationName (recNum, maxStaNamLen) |S1 ...
locationName (recNum, maxLocationLen) |S1 ...
QCT (QCcheckNum, QCcheckNameLen) |S1 ...
ICT (ICcheckNum, ICcheckNameLen) |S1 ...
latitude (recNum) float32 ...
longitude (recNum) float32 ...
elevation (recNum) float32 ...
I have tried multiple methods based on Help1 and Help2 to setup the boundaries which should be between latitude[20,53] and longitude[-131,-62]. The dataset can be accessed at NetCDF Data.
When I use the below, it says, "ValueError: dimensions or multi-index levels ['latitude', 'longitude'] do not exist"
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc',
decode_cf=False)
print(ds)
lat_bnds, lon_bnds = [20, 53], [-131, -62]
ds.sel(latitude=slice(*lat_bnds), longitude=slice(*lon_bnds))
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
When I try the below, it works through the data, but does not remove any data.
import xarray as xr
ds = xr.open_dataset('/home/awips/python-awips/ups/20181110_1600.nc', decode_cf=True)
ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
Any ideas?
python slice netcdf python-xarray
python slice netcdf python-xarray
edited Nov 15 '18 at 21:25
WxJack
asked Nov 15 '18 at 20:53
WxJackWxJack
13
13
Your second approach looks like the right way to solve this, for cases wherelatitudeandlongitudeare not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?
– shoyer
Nov 15 '18 at 21:20
I changed the code tods.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.
– WxJack
Nov 15 '18 at 21:49
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59
add a comment |
Your second approach looks like the right way to solve this, for cases wherelatitudeandlongitudeare not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?
– shoyer
Nov 15 '18 at 21:20
I changed the code tods.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.
– WxJack
Nov 15 '18 at 21:49
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59
Your second approach looks like the right way to solve this, for cases where
latitude and longitude are not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?– shoyer
Nov 15 '18 at 21:20
Your second approach looks like the right way to solve this, for cases where
latitude and longitude are not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?– shoyer
Nov 15 '18 at 21:20
I changed the code to
ds.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.– WxJack
Nov 15 '18 at 21:49
I changed the code to
ds.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.– WxJack
Nov 15 '18 at 21:49
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59
add a comment |
1 Answer
1
active
oldest
votes
Xarray operations usually return new objects instead of modifying objects inplace. So you need to assign the result of where to a new variable and save that instead, e.g.,
ds2 = ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
If I useds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')the size of the file increases dramatically and has tons of repeat variables that were not there before.
– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed bylatitudeandlongitude, no?
– delgadom
Nov 26 '18 at 2:30
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53327750%2fslice-netcdf-dataset%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Xarray operations usually return new objects instead of modifying objects inplace. So you need to assign the result of where to a new variable and save that instead, e.g.,
ds2 = ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
If I useds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')the size of the file increases dramatically and has tons of repeat variables that were not there before.
– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed bylatitudeandlongitude, no?
– delgadom
Nov 26 '18 at 2:30
add a comment |
Xarray operations usually return new objects instead of modifying objects inplace. So you need to assign the result of where to a new variable and save that instead, e.g.,
ds2 = ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
If I useds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')the size of the file increases dramatically and has tons of repeat variables that were not there before.
– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed bylatitudeandlongitude, no?
– delgadom
Nov 26 '18 at 2:30
add a comment |
Xarray operations usually return new objects instead of modifying objects inplace. So you need to assign the result of where to a new variable and save that instead, e.g.,
ds2 = ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
Xarray operations usually return new objects instead of modifying objects inplace. So you need to assign the result of where to a new variable and save that instead, e.g.,
ds2 = ds.where((-131 < ds.longitude) & (ds.longitude < -62)
& (20 < ds.latitude) & (ds.latitude < 53), drop=True)
ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')
answered Nov 16 '18 at 23:39
shoyershoyer
5,0391533
5,0391533
If I useds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')the size of the file increases dramatically and has tons of repeat variables that were not there before.
– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed bylatitudeandlongitude, no?
– delgadom
Nov 26 '18 at 2:30
add a comment |
If I useds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc')the size of the file increases dramatically and has tons of repeat variables that were not there before.
– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed bylatitudeandlongitude, no?
– delgadom
Nov 26 '18 at 2:30
If I use
ds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc') the size of the file increases dramatically and has tons of repeat variables that were not there before.– WxJack
Nov 18 '18 at 15:58
If I use
ds2 = ds.where((ds.latitude > 20) & (ds.latitude < 50) & (ds.longitude > -131) & (ds.longitude < -62), drop=True) ds2.to_netcdf(path='/home/awips/python-awips/ups/subset.nc') the size of the file increases dramatically and has tons of repeat variables that were not there before.– WxJack
Nov 18 '18 at 15:58
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
I don't know what's going on with the extra variables, but read this issue for a discussion of what's (probably) happening with file sizes: github.com/pydata/xarray/issues/1572
– shoyer
Nov 19 '18 at 16:10
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed by
latitude and longitude, no?– delgadom
Nov 26 '18 at 2:30
@shoyer this could also be related to the variables not indexed by latitude/longitude being broadcast against the where condition. The syntax you suggest should only be used for the variables in the Dataset that are indexed by
latitude and longitude, no?– delgadom
Nov 26 '18 at 2:30
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53327750%2fslice-netcdf-dataset%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Your second approach looks like the right way to solve this, for cases where
latitudeandlongitudeare not dimensions. I'm surprised it isn't removing any data -- are you sure sure there are records with longitude/latitude outside those bounds?– shoyer
Nov 15 '18 at 21:20
I changed the code to
ds.where((-95 < ds.longitude) & (ds.longitude < -80) & (30 < ds.latitude) & (ds.latitude < 35), drop=True). The file it creates is twice the size as the original, so something is wrong somewhere. The lat values are still there for -75.– WxJack
Nov 15 '18 at 21:49
@shoyer, I was able to remove the data by assigning it to a new variable, but I am not sure how I properly save the new data. What worked as latitude = ds.latitude.where((ds.latitude > 20) & (ds.latitude < 53), drop=True), longitude = ds.longitude.where((ds.longitude > -131) & (ds.longitude < -62), drop=True). Any further ideas?
– WxJack
Nov 16 '18 at 1:59