php - Iterate over Mongo results without running out of memory -


i need find keywords in name/description/tags, etc. of each document , remove them if found. i'm new mongo, i'm following similar script in existing codebase. first, mongocursor , fields we'll checking:

    /** @var mongocursor $products */     $products = $collection->find(         ['type' => ['$in' => ['phones', 'tablets']], 'supplier.is_awful' => ['$exists' => true]],         ['details.name' => true, 'details.description' => true]     ); 

then, iterate through each document, check each of properties values we're interested in:

/** @var \doctrine\odm\mongodb\documentmanager $manager */ $manager = new manager();  foreach ($products $product) {     // find objectionable words in content , remove these documents     foreach (["suckysucky's", "deuce", "a z z"] $word) {         if (false !== strpos(mb_strtolower($product['details']['name']), $word)           || false !== strpos(mb_strtolower($product['details']['description']), $word)) {                 $object = $manager->find(\app\product::class, $product['_id']);                 $manager->remove($object);         }     } } // persist db $manager->flush(); 

the problem database has hundreds of thousands of records, , looks iterating on mongocursor, memory usage goes , until runs out:

now @ (0) 20035632 @ (100) 24446048 @ (200) 32190312 @ (300) 36098208 @ (400) 42433656 @ (500) 45204376 @ (600) 50664808 @ (700) 54916888 @ (800) 59847312 @ (900) 65145808 @ (1000) 70764408 

is there way me iterate on mongocursor without running out of memory (i've tried unsetting various objects @ different points, no luck there)? alternatively, query can run directly in mongo? i've looked @ docs, , saw hope in $text, looks need have index there (i don't), , there can 1 text index per collection.

you don't need fulltext index find substring: right way using regex , return "_id" value, like:

$mongore = new mongoregex("/suckysucky's|deuce|a z z/i") $products = $collection->find(     ['type' => ['$in' => ['phones', 'tablets']],       'supplier.is_awful' => ['$exists' => true],      '$or': [['details.name' => $mongore],              ['details.description' => $mongore]]]     ['_id' => true] ); 

i'm not sure exact php syntax, key inclusive $or filter same mongodb regex on 2 fields.


Comments

Popular posts from this blog

c# - Binding a comma separated list to a List<int> in asp.net web api -

Delphi 7 and decode UTF-8 base64 -

html - Is there any way to exclude a single element from the style? (Bootstrap) -