Make a hint to SQLite that a particular column is always sorted









up vote
0
down vote

favorite












I have the following table in an SQLite database



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
`data` BLOB NOT NULL
) WITHOUT ROWID;

CREATE INDEX `time_index` ON `log`(`time`);


The index is created because the most frequent query is going to be



SELECT * FROM `log` WHERE `time` BETWEEN ? AND ?


Since the time is going to be always the current time when the new record is added, the index is not really required here. So I would like to "tell" the SQLite engine something like "The lines are going to be added with the 'time' column always having increasing value (similar to AUTO_INCREMENT), and if something goes wrong I will take all responsibility".



Is it possible at all?










share|improve this question





















  • Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
    – krokodilko
    Nov 12 at 8:14










  • When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
    – Alexander Solovets
    Nov 12 at 8:19










  • One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
    – Shawn
    Nov 12 at 8:24










  • Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
    – Alexander Solovets
    Nov 12 at 8:30














up vote
0
down vote

favorite












I have the following table in an SQLite database



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
`data` BLOB NOT NULL
) WITHOUT ROWID;

CREATE INDEX `time_index` ON `log`(`time`);


The index is created because the most frequent query is going to be



SELECT * FROM `log` WHERE `time` BETWEEN ? AND ?


Since the time is going to be always the current time when the new record is added, the index is not really required here. So I would like to "tell" the SQLite engine something like "The lines are going to be added with the 'time' column always having increasing value (similar to AUTO_INCREMENT), and if something goes wrong I will take all responsibility".



Is it possible at all?










share|improve this question





















  • Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
    – krokodilko
    Nov 12 at 8:14










  • When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
    – Alexander Solovets
    Nov 12 at 8:19










  • One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
    – Shawn
    Nov 12 at 8:24










  • Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
    – Alexander Solovets
    Nov 12 at 8:30












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have the following table in an SQLite database



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
`data` BLOB NOT NULL
) WITHOUT ROWID;

CREATE INDEX `time_index` ON `log`(`time`);


The index is created because the most frequent query is going to be



SELECT * FROM `log` WHERE `time` BETWEEN ? AND ?


Since the time is going to be always the current time when the new record is added, the index is not really required here. So I would like to "tell" the SQLite engine something like "The lines are going to be added with the 'time' column always having increasing value (similar to AUTO_INCREMENT), and if something goes wrong I will take all responsibility".



Is it possible at all?










share|improve this question













I have the following table in an SQLite database



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
`data` BLOB NOT NULL
) WITHOUT ROWID;

CREATE INDEX `time_index` ON `log`(`time`);


The index is created because the most frequent query is going to be



SELECT * FROM `log` WHERE `time` BETWEEN ? AND ?


Since the time is going to be always the current time when the new record is added, the index is not really required here. So I would like to "tell" the SQLite engine something like "The lines are going to be added with the 'time' column always having increasing value (similar to AUTO_INCREMENT), and if something goes wrong I will take all responsibility".



Is it possible at all?







sql sqlite sqlite3






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 12 at 7:34









Alexander Solovets

1,6621020




1,6621020











  • Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
    – krokodilko
    Nov 12 at 8:14










  • When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
    – Alexander Solovets
    Nov 12 at 8:19










  • One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
    – Shawn
    Nov 12 at 8:24










  • Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
    – Alexander Solovets
    Nov 12 at 8:30
















  • Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
    – krokodilko
    Nov 12 at 8:14










  • When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
    – Alexander Solovets
    Nov 12 at 8:19










  • One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
    – Shawn
    Nov 12 at 8:24










  • Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
    – Alexander Solovets
    Nov 12 at 8:30















Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
– krokodilko
Nov 12 at 8:14




Please explain what can go wrong, and what exactly you want to take responsibility for ? And what exactly would SQLite have to do with this information from you?
– krokodilko
Nov 12 at 8:14












When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
– Alexander Solovets
Nov 12 at 8:19




When you have an array of sorted values, you can find a particular item in the array by bisecting it rather than making a sequential search. I want to tell SQLite that the rows in the table described above are going to be naturally sorted by the 'time' column. And by 'go wrong' I mean the violation of the "sorted" rule. In this case SQLite might still do bisection even though the lines are not sorted anymore, and I would be completely fine with that.
– Alexander Solovets
Nov 12 at 8:19












One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
– Shawn
Nov 12 at 8:24




One thing: Your have your time column with an affinity of REAL, but a default value that is a TEXT string. Sqlite will happily let you do this, but it looks odd and might mislead people (Including yourself) about the format you intend to use with the column.
– Shawn
Nov 12 at 8:24












Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
– Alexander Solovets
Nov 12 at 8:30




Thank you, @Shawn. Actually, the default value in the real code is different, it computes UNIX time using the julianday function. I just wanted simplify my example.
– Alexander Solovets
Nov 12 at 8:30












2 Answers
2






active

oldest

votes

















up vote
1
down vote













You don't want a separate index. You want to declare the column to be the primary key:



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP PRIMARY KEY,
`data` BLOB NOT NULL
) WITHOUT ROWID;


This creates a single b-tree index for the log based on the primary key. In other databases, this structure would be called a "clustered index". You have probably already read the documentation but I'm referencing it anyway.






share|improve this answer




















  • That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
    – Alexander Solovets
    Nov 12 at 23:02

















up vote
1
down vote













You would have an issue, or not depending upon how you consider that you cannot use :-



CREATE TABLE `log` (
`time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
`data` BLOB NOT NULL
) WITHOUT ROWID;


because :-




Every WITHOUT ROWID table must have a PRIMARY KEY. An error is raised
if a CREATE TABLE statement with the WITHOUT ROWID clause lacks a
PRIMARY KEY.
Clustered Indexes and the WITHOUT ROWID Optimization




So you might as well make the time column the PRIMARY KEY.





but the problem is that the precision of REAL is not enough to handle
microsecond resolution, and thus two adjacent records may have the
same time value which would violate the PRIMARY KEY constraint.




Then you could use a composite PRIMARY KEY where the precision required is satisfied by multiple columns (a second column would likely more than suffice) perhaps along the lines of :-



CREATE TABLE log (
time_datepart INTEGER,
time_microsecondpart,
data BLOB NOt NULL,
PRIMARY KEY (time_datepart,time_microsecondpart)
) WITHOUT ROWID;


The time_microsecondpart column needn't necessarily be microseconds it could be a counter derived from another table similar to how the sqlite_sequence table is utilised when AUTOINCREMENT is utilised (less the need for the column that holds the name of the table that a row is attached to).






share|improve this answer






















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257624%2fmake-a-hint-to-sqlite-that-a-particular-column-is-always-sorted%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote













    You don't want a separate index. You want to declare the column to be the primary key:



    CREATE TABLE `log` (
    `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP PRIMARY KEY,
    `data` BLOB NOT NULL
    ) WITHOUT ROWID;


    This creates a single b-tree index for the log based on the primary key. In other databases, this structure would be called a "clustered index". You have probably already read the documentation but I'm referencing it anyway.






    share|improve this answer




















    • That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
      – Alexander Solovets
      Nov 12 at 23:02














    up vote
    1
    down vote













    You don't want a separate index. You want to declare the column to be the primary key:



    CREATE TABLE `log` (
    `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP PRIMARY KEY,
    `data` BLOB NOT NULL
    ) WITHOUT ROWID;


    This creates a single b-tree index for the log based on the primary key. In other databases, this structure would be called a "clustered index". You have probably already read the documentation but I'm referencing it anyway.






    share|improve this answer




















    • That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
      – Alexander Solovets
      Nov 12 at 23:02












    up vote
    1
    down vote










    up vote
    1
    down vote









    You don't want a separate index. You want to declare the column to be the primary key:



    CREATE TABLE `log` (
    `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP PRIMARY KEY,
    `data` BLOB NOT NULL
    ) WITHOUT ROWID;


    This creates a single b-tree index for the log based on the primary key. In other databases, this structure would be called a "clustered index". You have probably already read the documentation but I'm referencing it anyway.






    share|improve this answer












    You don't want a separate index. You want to declare the column to be the primary key:



    CREATE TABLE `log` (
    `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP PRIMARY KEY,
    `data` BLOB NOT NULL
    ) WITHOUT ROWID;


    This creates a single b-tree index for the log based on the primary key. In other databases, this structure would be called a "clustered index". You have probably already read the documentation but I'm referencing it anyway.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Nov 12 at 12:38









    Gordon Linoff

    754k35289397




    754k35289397











    • That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
      – Alexander Solovets
      Nov 12 at 23:02
















    • That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
      – Alexander Solovets
      Nov 12 at 23:02















    That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
    – Alexander Solovets
    Nov 12 at 23:02




    That's the solution that I naturally discovered myself, but the problem is that the precision of REAL is not enough to handle microsecond resolution, and thus two adjacent records may have the same time value which would violate the PRIMARY KEY constraint. Is there a technique to work around this problem? Or is having a separate index my only solution?
    – Alexander Solovets
    Nov 12 at 23:02












    up vote
    1
    down vote













    You would have an issue, or not depending upon how you consider that you cannot use :-



    CREATE TABLE `log` (
    `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `data` BLOB NOT NULL
    ) WITHOUT ROWID;


    because :-




    Every WITHOUT ROWID table must have a PRIMARY KEY. An error is raised
    if a CREATE TABLE statement with the WITHOUT ROWID clause lacks a
    PRIMARY KEY.
    Clustered Indexes and the WITHOUT ROWID Optimization




    So you might as well make the time column the PRIMARY KEY.





    but the problem is that the precision of REAL is not enough to handle
    microsecond resolution, and thus two adjacent records may have the
    same time value which would violate the PRIMARY KEY constraint.




    Then you could use a composite PRIMARY KEY where the precision required is satisfied by multiple columns (a second column would likely more than suffice) perhaps along the lines of :-



    CREATE TABLE log (
    time_datepart INTEGER,
    time_microsecondpart,
    data BLOB NOt NULL,
    PRIMARY KEY (time_datepart,time_microsecondpart)
    ) WITHOUT ROWID;


    The time_microsecondpart column needn't necessarily be microseconds it could be a counter derived from another table similar to how the sqlite_sequence table is utilised when AUTOINCREMENT is utilised (less the need for the column that holds the name of the table that a row is attached to).






    share|improve this answer


























      up vote
      1
      down vote













      You would have an issue, or not depending upon how you consider that you cannot use :-



      CREATE TABLE `log` (
      `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `data` BLOB NOT NULL
      ) WITHOUT ROWID;


      because :-




      Every WITHOUT ROWID table must have a PRIMARY KEY. An error is raised
      if a CREATE TABLE statement with the WITHOUT ROWID clause lacks a
      PRIMARY KEY.
      Clustered Indexes and the WITHOUT ROWID Optimization




      So you might as well make the time column the PRIMARY KEY.





      but the problem is that the precision of REAL is not enough to handle
      microsecond resolution, and thus two adjacent records may have the
      same time value which would violate the PRIMARY KEY constraint.




      Then you could use a composite PRIMARY KEY where the precision required is satisfied by multiple columns (a second column would likely more than suffice) perhaps along the lines of :-



      CREATE TABLE log (
      time_datepart INTEGER,
      time_microsecondpart,
      data BLOB NOt NULL,
      PRIMARY KEY (time_datepart,time_microsecondpart)
      ) WITHOUT ROWID;


      The time_microsecondpart column needn't necessarily be microseconds it could be a counter derived from another table similar to how the sqlite_sequence table is utilised when AUTOINCREMENT is utilised (less the need for the column that holds the name of the table that a row is attached to).






      share|improve this answer
























        up vote
        1
        down vote










        up vote
        1
        down vote









        You would have an issue, or not depending upon how you consider that you cannot use :-



        CREATE TABLE `log` (
        `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `data` BLOB NOT NULL
        ) WITHOUT ROWID;


        because :-




        Every WITHOUT ROWID table must have a PRIMARY KEY. An error is raised
        if a CREATE TABLE statement with the WITHOUT ROWID clause lacks a
        PRIMARY KEY.
        Clustered Indexes and the WITHOUT ROWID Optimization




        So you might as well make the time column the PRIMARY KEY.





        but the problem is that the precision of REAL is not enough to handle
        microsecond resolution, and thus two adjacent records may have the
        same time value which would violate the PRIMARY KEY constraint.




        Then you could use a composite PRIMARY KEY where the precision required is satisfied by multiple columns (a second column would likely more than suffice) perhaps along the lines of :-



        CREATE TABLE log (
        time_datepart INTEGER,
        time_microsecondpart,
        data BLOB NOt NULL,
        PRIMARY KEY (time_datepart,time_microsecondpart)
        ) WITHOUT ROWID;


        The time_microsecondpart column needn't necessarily be microseconds it could be a counter derived from another table similar to how the sqlite_sequence table is utilised when AUTOINCREMENT is utilised (less the need for the column that holds the name of the table that a row is attached to).






        share|improve this answer














        You would have an issue, or not depending upon how you consider that you cannot use :-



        CREATE TABLE `log` (
        `time` REAL NOT NULL DEFAULT CURRENT_TIMESTAMP,
        `data` BLOB NOT NULL
        ) WITHOUT ROWID;


        because :-




        Every WITHOUT ROWID table must have a PRIMARY KEY. An error is raised
        if a CREATE TABLE statement with the WITHOUT ROWID clause lacks a
        PRIMARY KEY.
        Clustered Indexes and the WITHOUT ROWID Optimization




        So you might as well make the time column the PRIMARY KEY.





        but the problem is that the precision of REAL is not enough to handle
        microsecond resolution, and thus two adjacent records may have the
        same time value which would violate the PRIMARY KEY constraint.




        Then you could use a composite PRIMARY KEY where the precision required is satisfied by multiple columns (a second column would likely more than suffice) perhaps along the lines of :-



        CREATE TABLE log (
        time_datepart INTEGER,
        time_microsecondpart,
        data BLOB NOt NULL,
        PRIMARY KEY (time_datepart,time_microsecondpart)
        ) WITHOUT ROWID;


        The time_microsecondpart column needn't necessarily be microseconds it could be a counter derived from another table similar to how the sqlite_sequence table is utilised when AUTOINCREMENT is utilised (less the need for the column that holds the name of the table that a row is attached to).







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Nov 13 at 1:43

























        answered Nov 12 at 10:49









        MikeT

        14.3k102441




        14.3k102441



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257624%2fmake-a-hint-to-sqlite-that-a-particular-column-is-always-sorted%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Top Tejano songwriter Luis Silva dead of heart attack at 64

            ReactJS Fetched API data displays live - need Data displayed static

            政党