multithreading - What is most efficient Python IPC mechanism for plotting real-time serial data? -

what fastest python mechanism getting data read off of serial port, separate process plotting data?

i plotting eeg data in real-time read off of serial port. serial port reading , packet unpacking code works fine, in if read , store data, later plot stored data, looks great. this:

note: device generates test sine wave debugging

i using pyqtgraph plotting. updating plot in same process read serial data in not option because slight delay between serial read() calls causes serial buffer overflow , bad check-sums ensue. pyqtgraph has provisions rendering graph on separate process, great, bottle-neck seems in inter-process communication. have tried various configuration of pipe() , queue(), of result in laggy, flickering graph updates. far, smoothest, consistent method of getting new values serial port graph seems through shared memory, so:

from pyqtgraph.qt import qtgui import pyqtgraph pg multiprocessing import process, array, value, pipe serial_interface import eeg64board collections import deque  def serialloop(arr):     eeg = eeg64board(port='/dev/ttyacm0')     eeg.openserial()      eeg.sendtest('1')        #tells eeg device start sending data     while true:         data = eeg.readeeg() #returns array of 8 latest values, 1 per channel         if data != false:    #returns false if bad checksum             val.value = data[7]   val = value('d',0.0) q = deque([],500)  def graphloop():     global val,q     plt = pg.plot(q)     while true:         q.append(val.value)         plt.plot(q,clear=true)         qtgui.qapplication.processevents()  serial_proc = process(target=serialloop, args=(val,), name='serial_proc') serial_proc.start()  try:     while true:         graphloop() except keyboardinterrupt:     print('interrupted') 

the above code performs real-time plotting pulling latest value recorded serialloop , appending deque. while plot updates smoothly, grabbing 1 in 4 values, seen in resulting plot:

so, multi-process or thread structure recommend, , form of ipc should used between them?

update:

i receiving 2,000 samples per second. thinking if update display @ 100 fps , add 20 new samples per frame should good. best python multithreading mechanism implementing this?

this may not efficient, following code achieves 100 fps 1 plot, or 20 fps 8 plots. idea simple: share array, index, , lock. serial fills array , increments index while has lock, plotting process periodically grabs of new values array , decrements index, again, under lock.

from pyqtgraph.qt import qtgui import pyqtgraph pg multiprocessing import process, array, value, lock serial_interface import eeg64board collections import deque  def serialloop(arr,idx,lock):     eeg = eeg64board(port='/dev/ttyacm0')     eeg.openserial()      eeg.sendtest('1')        #tells eeg device start sending data     while true:         data = eeg.readeeg() #returns array of 8 latest values, 1 per channel         if data != false:    #returns false if bad checksum             lock.acquire()             in range(8):                 arr[i][idx.value] = data[i]              idx.value += 1             lock.release()     eeg.sendtest('2')   arr = [array('d',range(1024)) in range(8)] idx = value('i', 0) q = [deque([],500) in range(8)] iq = deque([],500) lock = lock()  lastupdate = pg.ptime.time() avgfps = 0.0  def graphloop():     global val,q,lock,arr,iq, lastupdate, avgfps     win = pg.graphicswindow()     plt = list()     in range(8):         plt += [win.addplot(row=(i+1), col=0, colspan=3)]     #iplt = pg.plot(iq)     counter = 0     while true:         lock.acquire()         #time.sleep(.01)         in range(idx.value):             j in range(8):                 q[j].append(arr[j][i])                 idx.value = 0         lock.release()         in range(8):             plt[i].plot(q[i],clear=true)         qtgui.qapplication.processevents()         counter += 1          = pg.ptime.time()         fps = 1.0 / (now - lastupdate)         lastupdate =         avgfps = avgfps * 0.8 + fps * 0.2  serial_proc = process(target=serialloop, args=(arr,idx,lock), name='serial_proc') serial_proc.start()  graphloop()  serial_proc.terminate()