Rails Sunspot/Solr Index Out of Sync
I'm using the sunspot-rails gem for searching and filtering but am running into issues where Solr queries are not returning the correct results. For example, say I want to retrieve all the records of a model with state == Processing, and I do the following Sunspot search:
MyModel.search do
# pagination and ordering stuff
...
with('state', 'Processing')
...
end
Most of the time, this returns the correct results. Sometimes (and so far only on production, I can't duplicate the issue locally) the query will return records with state == In Review. If I do a regular ActiveRecord MyModel.where(state: 'Processing') I always get the correct results.
I thought it might have to do with my solrconfig.yml file but changing those params haven't seemed to change anything. Relevant portion of that file:
<autoCommit>
<maxTime>15000</maxTime>
<maxDocs>1000</maxDocs>
<openSearcher>true</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>5000</maxTime>
</autoSoftCommit>
Does anyone have any pointers for why changes in my db aren't reflected in the Solr index, or how I can debug/log what's going on? This is on a small internal app with maybe 100 users or so. I shouldn't have to reindex Solr daily to keep the results up to date.
Thanks.
ruby-on-rails solr sunspot sunspot-rails
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
add a comment |
I'm using the sunspot-rails gem for searching and filtering but am running into issues where Solr queries are not returning the correct results. For example, say I want to retrieve all the records of a model with state == Processing, and I do the following Sunspot search:
MyModel.search do
# pagination and ordering stuff
...
with('state', 'Processing')
...
end
Most of the time, this returns the correct results. Sometimes (and so far only on production, I can't duplicate the issue locally) the query will return records with state == In Review. If I do a regular ActiveRecord MyModel.where(state: 'Processing') I always get the correct results.
I thought it might have to do with my solrconfig.yml file but changing those params haven't seemed to change anything. Relevant portion of that file:
<autoCommit>
<maxTime>15000</maxTime>
<maxDocs>1000</maxDocs>
<openSearcher>true</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>5000</maxTime>
</autoSoftCommit>
Does anyone have any pointers for why changes in my db aren't reflected in the Solr index, or how I can debug/log what's going on? This is on a small internal app with maybe 100 users or so. I shouldn't have to reindex Solr daily to keep the results up to date.
Thanks.
ruby-on-rails solr sunspot sunspot-rails
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44
add a comment |
I'm using the sunspot-rails gem for searching and filtering but am running into issues where Solr queries are not returning the correct results. For example, say I want to retrieve all the records of a model with state == Processing, and I do the following Sunspot search:
MyModel.search do
# pagination and ordering stuff
...
with('state', 'Processing')
...
end
Most of the time, this returns the correct results. Sometimes (and so far only on production, I can't duplicate the issue locally) the query will return records with state == In Review. If I do a regular ActiveRecord MyModel.where(state: 'Processing') I always get the correct results.
I thought it might have to do with my solrconfig.yml file but changing those params haven't seemed to change anything. Relevant portion of that file:
<autoCommit>
<maxTime>15000</maxTime>
<maxDocs>1000</maxDocs>
<openSearcher>true</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>5000</maxTime>
</autoSoftCommit>
Does anyone have any pointers for why changes in my db aren't reflected in the Solr index, or how I can debug/log what's going on? This is on a small internal app with maybe 100 users or so. I shouldn't have to reindex Solr daily to keep the results up to date.
Thanks.
ruby-on-rails solr sunspot sunspot-rails
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
I'm using the sunspot-rails gem for searching and filtering but am running into issues where Solr queries are not returning the correct results. For example, say I want to retrieve all the records of a model with state == Processing, and I do the following Sunspot search:
MyModel.search do
# pagination and ordering stuff
...
with('state', 'Processing')
...
end
Most of the time, this returns the correct results. Sometimes (and so far only on production, I can't duplicate the issue locally) the query will return records with state == In Review. If I do a regular ActiveRecord MyModel.where(state: 'Processing') I always get the correct results.
I thought it might have to do with my solrconfig.yml file but changing those params haven't seemed to change anything. Relevant portion of that file:
<autoCommit>
<maxTime>15000</maxTime>
<maxDocs>1000</maxDocs>
<openSearcher>true</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>5000</maxTime>
</autoSoftCommit>
Does anyone have any pointers for why changes in my db aren't reflected in the Solr index, or how I can debug/log what's going on? This is on a small internal app with maybe 100 users or so. I shouldn't have to reindex Solr daily to keep the results up to date.
Thanks.
ruby-on-rails solr sunspot sunspot-rails
ruby-on-rails solr sunspot sunspot-rails
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
asked Nov 16 '18 at 0:45
Matt DavisMatt Davis
14
14
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44
add a comment |
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53329864%2frails-sunspot-solr-index-out-of-sync%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53329864%2frails-sunspot-solr-index-out-of-sync%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What's the time difference between the query being made and the document being shown? Does a document go from Processing (i.e. is the status in Solr processing) to In review? Can that change happen in the second or two that an update may take?
– MatsLindh
Nov 16 '18 at 9:32
This might be unrelated to Solr. The index being synced or not depends first and foremost on how indexing is triggered after some changes have been detected. If you manually change the state of a record, how is this reflected on the Solr side ? What is the logic triggering a Solr update request ?
– EricLavault
Nov 16 '18 at 11:12
A record's state can change to one of 10 fixed strings, and can go from one state to any other. Typically, a user clicks a button that updates a record's state, and then several days later runs the Solr query above and find the record they changed is still marked with its old state in the query results, but on the record's show page it will have the correct state. The Solr update triggers are handled by the Sunspot gem, and AFAIK it should update every time an AR update is called. I thought there might be a bug with Sunspot but haven't found anyone else with this same issue using Sunspot.
– Matt Davis
Nov 16 '18 at 17:44