mysql - Python MySQLdb crashing randomly without errors -


i'm using python mysqldb perform calculations on large 75m row database. basic program flow iterate through every row, process data, use cursor put new data in row. processing complex pure sql.

db1 = mysqldb.connect(host=host, user=username, passwd=password, db=database) db1.ping(true) cursor1 = db1.cursor() db = mysqldb.connect(host=host, user=username, passwd=password, db=database) db2.ping(true) cursor2 = db2.cursor()  cursor1.execute("select * table") row in cursor1:     cursor2.execute("select * table")     new_data    = process_stuff(row)      cursor2.execute("update table set column = '%f'", (new_data))  db2.commit() db1.close() db2.close()

this works, usually. though, silently quits no error message. sometimes, not always, give "mysql has gone away" error.

it's rather slow. know it's large database, takes 24 hours process everything. it's decent machine 24 gb of ram, should take long? i've spent hours fiddling my.conf, no real gain.


-

Comments

Post a Comment

Popular posts from this blog

swift - Button on Table View Cell connected to local function -

i have table view cell loaded data parse request. loads "friend requests", showing requesting friends name , 2 buttons, "accept" , "reject" here cellforrowatindexpath func: func tableview(tableview: uitableview, cellforrowatindexpath indexpath: nsindexpath) -> uitableviewcell { let cell:notifications_jlr_cell = self.tableview.dequeuereusablecellwithidentifier("cell") as! notifications_jlr_cell cell.txtrequestdesc.text = searchresults[indexpath.row].requesteename + " has requested friend" cell.btnapprove_outlet.addtarget(self, action:selector("approve"), forcontrolevents: uicontrolevents.touchupinside) return cell } the table loads fine, , addtarget line, once "approve" button clicked, runs func: func approve() { println("approve") } however, when fetching parse data, have set in array, objectid friend request, want refer in approve() function how can pass objectid ...
Read more

dns - Dokku server hosts two sites with TLD's, both domains are landing on only one app -

i have 2 simple dokku sites on 1 server. first static html site, , second simple rails app. each app has own tld. sake of argument give them domains: static.com , rails.com. ive used dokku domains plugin set both naked , 'www' domains each site. domain names purchased , setup through namecheap the namecheap domains each have 2 entries: @ | http://www.static.com | url redirect www | static.server.com | cname(alias) @ | http://www.rails.com | url redirect www | rails.server.com | cname(alias) i encountered this issue on dokku github page indicate can overcome using naked domain app name when creating original git remote. git remote add dokku dokku@server.com:static.com however, issue closed 2 years ago, , same result seems accomplished domains plugin. im wondering if should restart point both apps? so far above steps has resulted in traffic both domains being sent static.com. has encountered before? t...
Read more

c# - ajax - How to receive data both html and json from server? -

i have referenced this topic , doesn't solve problem. i'm using mvc 5 c#. have used code receive data server data type json . $.ajax({ url: "/mycontroller/myaction", type: "post", datatype: "json", success: function (data) { if (data.result) { alert('successfull'); } else { alert(data.ex); } } }); and controller code: [httppost] public actionresult myaction() { try { return json(new { result = "true", ex = "" }); } catch (exception e) { return json(new { result = "false", ex = e.message }); } } i use way data type html: $.ajax({ url: "/mycontroller/myaction", type: "post", datatype: "html", success: function (data) { $(".mydiv").append(data); } }); and controller should be: [httppost] public actionresult myaction() { return partialview("_...
Read more