File-related-handlers are still left open after shut down

Oct 7, 2015 at 10:10 AM
After I finish RaptorDB, I want to delete the directory and clean up the mess I did. When I am doing that, it reports that there is one file still in use:
The process cannot access the file 'RDB\Data\Fulltext\fulltext_hoot.mgbmp' because it is being used by another process.
The process cannot access the file 'RDB\Data\Fulltext\fulltext_hoot.mgbmp' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\NMT_ID.mgbmr' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\TmElementInfoView.mgdat' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\TmElementInfoView.mgdat' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\NMT_ID.mgidx' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\TmElementInfoView.mgrec' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\DLW_TEXT.strings' because it is being used by another process.
The process cannot access the file 'RDB\Views\TmElementInfoView\TME_POS3.mgbmp' because it is being used by another process.
This happens also for other views in the same RaptorDB instance. The directory delete is run after RaptorDB.Shutdown method is called.

If case, I "schedule" the deletion after 5 minutes - but it is still the same. If fails, reschedule again - and the same happens. I am worried this might be a leak and if I use this for long run, I may end up using a lot of file handlers left in memory ... is there something I can do about this?
Coordinator
Oct 7, 2015 at 10:53 AM
That is strange, I will check this.
Oct 7, 2015 at 1:58 PM
I tried with:
    [RegisterView]
    public class TmElementInfoView : RaptorDB.View<TmElementInfo>
    {

        #region constructor

        public TmElementInfoView()
        {
            this.Name = "TmElementInfoView";
            this.Description = "Storage for #TMB_ELEMENT elements";
            this.isPrimaryList = true;
            this.BackgroundIndexing = false;
            this.ConsistentSaveToThisView = true;
            this.Schema = typeof(TmElementInfoSchema);
            this.TransactionMode = false;
            this.Version = 1;
            this.DeleteBeforeInsert = false;
            this.Mapper = (api, docid, doc) =>
            {
                api.EmitObject(docid, doc);
            };
        }

        #endregion
   }
hoping there are no background threads that keep handlers opened to the repository - but it is still the same.
Oct 12, 2015 at 11:09 AM
It looks like this solved the problem for data (Hoot.cs):
        public void Shutdown()
        {
            lock (_lock)
            {
                InternalSave();

                if (null != _bitmaps)
                {
                    _bitmaps.Commit(Global.FreeBitmapMemoryOnSave);
                    _bitmaps.Shutdown();
                    _bitmaps = null;
                }

                if (_docMode)
                {
                    _docs.Shutdown();
                    _deleted.Shutdown();
                }
            }
        }
Please review it and let me know if it's ok or it doesn't have any side effects. Meanwhile, I'll check the indexes on views ...
Coordinator
Oct 12, 2015 at 11:14 AM
Nice catch!

It's all good :)
Coordinator
Oct 12, 2015 at 6:37 PM
You need to add the following to hoot.cs InternalSave()
            if (_bitmaps != null)
                _bitmaps.Commit(false);
Also you can omit the _bitmaps.Commit() in Shutdown()
Oct 13, 2015 at 8:19 AM
Edited Oct 13, 2015 at 8:26 AM
I already had the commit call in InternalSave - it's true, it was not guarded by the null-pointer-check. I still have the file-in-use exception for views.
Oct 13, 2015 at 11:51 AM
Edited Oct 13, 2015 at 11:52 AM
Getting closer:

Out of 6 views opened, only 3 have this issue. The logging looks like this:
2015-10-13 12:34:57|DEBUG|28|RaptorDB.MGIndex`1|SaveIndex| Total pages = 1
2015-10-13 12:34:57|ERROR|28|RaptorDB.Views.ViewHandler|Shutdown| System.NullReferenceException: Object reference not set to an instance of an object.
   at RaptorDB.IndexFile`1.SavePage(Page`1 node) in c:\code\RaptorDB\Indexes\IndexFile.cs:line 328
   at RaptorDB.MGIndex`1.SaveIndex() in c:\code\RaptorDB\Indexes\MGIndex.cs:line 206
   at RaptorDB.TypeIndexes`1.RaptorDB.IIndex.Shutdown() in c:\code\RaptorDB\Indexes\Indexes.cs:line 54
   at RaptorDB.Views.ViewHandler.Shutdown() in c:\code\RaptorDB\Views\ViewHandler.cs:line 483
2015-10-13 12:34:57|DEBUG|28|RaptorDB.Views.ViewManager|ShutDown|  shutting down view : TmElementInfoView
I am not sure right now if the lines are still matching with original code (I may have added some tracing code), but here is where it throws the exception:
internal void SavePage(Page<T> node)
{
     if (_externalStrings == false) 
          kk = _T.GetBytes(kp);
}
where _T is null.

Any idea on how to fix this?

Thanks!
Oct 13, 2015 at 12:51 PM
This has been added to DataTypes.cs:
public static IGetBytes<T> ByteHandler()
{
       else if (type == typeof(ushort)) return (IGetBytes<T>)new ushort_handler<T>();
}

internal class uint_handler<T> : IGetBytes<uint>
{
    public byte[] GetBytes(uint obj)
    {
        return Helper.GetBytes(obj, false);
    }
    public uint GetObject(byte[] buffer, int offset, int count)
    {
        return (uint)Helper.ToInt32(buffer, offset);
    }
}
Could you please review? I am a little bit worried about SafeDictionary.Helper.GetBytes - 4 bytes vs 2 bytes - but a similar thing was done for uint (goes to GetBytes(long, bool) ).

After fixing this, the files were successfully removed. However, I noticed some uncaught casting-exceptions in Visual Studio's output window:
A first chance exception of type 'System.InvalidCastException' occurred in RaptorDB.dll
Are there any other (casting-)limitations we should be aware of?

Thanks.