Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about usage #5

Open
ghost opened this issue Jun 11, 2017 · 0 comments
Open

Question about usage #5

ghost opened this issue Jun 11, 2017 · 0 comments

Comments

@ghost
Copy link

ghost commented Jun 11, 2017

Hello,

i wanted to ask you 2 questions. I am using Libspatialindex in C++, however no matter what i do, as soon as i use the query functions(doesnt matter if nearestNeighborQuery, queryStrategy or intersectsWithQuery) with my own defined visitor, memory is still allocated even after calling delete, but there is no memory leak anywhere. Did you guys also experienced this issue? I am attaching simple program demonstrating the issue. Problem is when i am running my code inside deamon, requests which traverse the tree a lot allocated hundreds of MB of memory and i can`t figure out what is going on,because there is nowhere memory leaking.

#include <spatialindex/SpatialIndex.h>
#include <cstring>

class MemoryUsage
{
    //VmRSS is the measure of how much psysical RAM the process is actually using (allocated and used)
    double m_vmrss = 0;

    //Part of the RSS is allocated in real memory blocks (other than mapped doubleo a file or device).
    //This is anonymous memory (anon-rss) and there is also RSS memory blocks that are mapped doubleo devices and files (file-rss)
    double m_rssanon = 0;
    double m_rssfile = 0;
    double m_rssshmem = 0; //shared memory

    static double parse_number(const std::string &line)
    {
        const auto f = line.find_first_of("0123456789");
        const auto l = line.find_last_of("0123456789");
        return std::atoi(line.substr(f, l - f + 1).c_str());
    }

  public:
    /**
         * Get the memory usage for the current process. The constructor will
         * get the memory usage
         */
    MemoryUsage()
    {
        monitor_usage();
    }

    //this function checks /proc/self/status, parses the values and updates member values if current value is higher
    void monitor_usage()
    {
        static const char *filename = "/proc/self/status";
        std::ifstream status_file(filename);

        if (status_file.is_open())
        {
            std::string line;
            while (!status_file.eof())
            {
                std::getline(status_file, line);
                if (line.substr(0, 5) == "VmRSS")
                {
                    double current_vmrss = parse_number(line);
                    if (current_vmrss > m_vmrss)
                    {
                        m_vmrss = current_vmrss;
                    }
                }
                if (line.substr(0, 7) == "RssAnon")
                {
                    double current_rssanon = parse_number(line);
                    if (current_rssanon > m_rssanon)
                    {
                        m_rssanon = current_rssanon;
                    }
                }
                if (line.substr(0, 7) == "RssFile")
                {
                    double current_rssfile = parse_number(line);
                    if (current_rssfile > m_rssfile)
                    {
                        m_rssfile = current_rssfile;
                    }
                }
                if (line.substr(0, 8) == "RssShmem")
                {
                    double current_rssshmem = parse_number(line);
                    if (current_rssshmem > m_rssshmem)
                    {
                        m_rssshmem = current_rssshmem;
                    }
                }
            }
        }
    }

    // Return total memory usage in MBytes
    double vm_rss() const
    {
        return m_vmrss / 1024;
    }

    // Return memory usage not mapped to files in MBytes
    double rss_anon() const
    {
        return m_rssanon / 1024;
    }

    // Return memory usage for files and devices mapping in MBytes
    double rss_file() const
    {
        return m_rssfile / 1024;
    }

    //Return memory usage for shared memory in MBytes
    double rss_shared() const
    {
        return m_rssshmem / 1024;
    }
};

struct RTreeData
{
    uint32_t source;
};

using namespace std;
namespace si = SpatialIndex;

int main()
{
    {
        //create in memory r-tree
        si::IStorageManager *memory_manager = si::StorageManager::createNewMemoryStorageManager();
        double const fill_factor = 0.7;      //default 0.7 (R*), 0.0-0.5 for linear/quadratic
        uint32_t const index_capacity = 100; //default 100
        uint32_t const leaf_capacity = 100;  // default: 100
        uint32_t const dimension = 2;
        si::RTree::RTreeVariant variant = si::RTree::RV_RSTAR;
        si::id_type index_id;
        si::ISpatialIndex *rtree = si::RTree::createNewRTree(*memory_manager, fill_factor, index_capacity, leaf_capacity, dimension, variant, index_id);

        //fill up tree with 10k test points
        {
            vector<pair<double, double>> points;

            //generate random points
            double lat = 40.0;
            double lon = 17.0;
            for (int i = 0; i < 10000; ++i)
            {
                points.push_back(make_pair(lat, lon));
                lat += 0.0001;
                lon += 0.0001;
            }

            uint32_t i = 1;
            for (auto &p : points)
            {
                double plow[] = {get<0>(p), get<1>(p)};
                double phigh[] = {get<0>(p), get<1>(p)};
                si::Region mbr(plow, phigh, 2);
                si::id_type item_id(i);
                RTreeData data{i};
                rtree->insertData(sizeof(data), reinterpret_cast<byte *>(&data), mbr, item_id);
                i++;
            }
        }

        MemoryUsage ms;
        cout << "Allocated memory(except files) is " << ms.rss_anon() << " MB" << endl;

        //try to query all 10k points
        {
            TestPoiVisitor nearest_neighbor_visitor;
            double q_coords[] = {48.145584, 17.1271079};
            si::Point q(q_coords, 2);
            uint32_t k = 10000;
            rtree->nearestNeighborQuery(k, q, nearest_neighbor_visitor);
            vector<tuple<si::Region, uint32_t>> nearestNeighborCandidates = nearest_neighbor_visitor.getNearestCandidates();
            cout << "Found " << to_string(nearestNeighborCandidates.size()) << " results" << endl;
        }

        delete rtree;
        delete memory_manager;

        ms.monitor_usage();
        cout << "Allocated memory(except files) is " << ms.rss_anon() << " MB" << endl;
    }
return 0;
}

Output on this small dataset is

Allocated memory(except files) is 1.54688 MB
Found 10000 results
Allocated memory(except files) is 2.72656 MB

Second question, i was browsing through your code, but i have no idea on how to use it. There is nowhere a schema on how the binary data for inserting data/query should look like. Would you be so kind and write me simple example based on my previous code on how can i create index, add points and query?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants