How to delete old version index file

[ashokblend]

In our project we ingest records every minute. Hence we do refresh index on incremental records every minute. This activity creates lot of version of index.

Is there any configuration to keep only last "N" version of index ?

[EJ Song (sezruby)]

There's a PR for vacuumIndex which allows to remove all outdated versions of the index - #479 - though it removes all index data except for the latest version.

paryoja Is it possible to extend vacuumIndex to support the last N version of index? WDYT?

[ashokblend]

I don't have any use case to retain last N index. I just wanted to cleanup stale indexes. I think mentioned PR will suffice that. Thanks EJ Song (@sezruby)

[paryoja]

EJ Song (@sezruby) Yes, I think I can find out last N version of index files by scanning the index logs.
However, if the data is continuously increasing without deletion, I think it is possible to most index files are still in use.

[ashokblend]

having large no of stale version causes performance bottleneck? I had face this issue when there was around 500's of version file. hs.explain query was taking lot of time.