Discuss Scratch

chrowe
Scratcher
6 posts

Export/pipe data out of Scratch

I am looking for a way to send values from Scratch to some form of external destination. This could be a local file or a web-based API or anything like that. It seems like there are lots of ways to get data from real-world sensors into Scratch and lots of ways to use those values to control connected devices but I don't see any way to send that data to an external location.

My use case is for storing GrovePi sensor values in the cloud.

Last edited by chrowe (Feb. 23, 2018 06:57:20)

myeducate
Scratcher
500+ posts

Export/pipe data out of Scratch

Check out scratchx.org

INACTIVE
SPA Member and Assosiate - Creator and overlord of ScratchNetwork - 700+ Forum Posts - Web and Desktop Dev - Fluent in VB, PHP and HTML. I'm okay at CSS and Javascript but am still learning. Sig written in PHP using the picture libary. Firebase is fun.

SimpleScratch
Scratcher
500+ posts

Export/pipe data out of Scratch

Hi
I'm just in the process of finishing off a GrovePi Scratch 2 extension.

What sort of cloud storage are you wanting to use?


In the meantime - I had a bit of a play and I've added a simple file write that outputs all sensor values to a text file on the Pi

I then put together a Node-RED flow that reads the file and posts each sensor value to its own topic on an MQTT server

Last edited by SimpleScratch (Feb. 26, 2018 16:43:14)

chrowe
Scratcher
6 posts

Export/pipe data out of Scratch

Great to hear that @SimSimpleScratch

I think MQTT and REST are the most common protocols. How easy is it to make a general purpose component with configuration options. The Node-RED MQTT nodes is a good example https://flows.nodered.org/node/node-red-contrib-mqtt-broker
chrowe
Scratcher
6 posts

Export/pipe data out of Scratch

Do you have code that I can test out?
SimpleScratch
Scratcher
500+ posts

Export/pipe data out of Scratch

How easy is it to make a general purpose component with configuration options.
I'll see if I can make a standalone MQTT extension
(I'm not a javascript expert so it might take me a little time but hopefully I'll find a library to use)
SimpleScratch
Scratcher
500+ posts

Export/pipe data out of Scratch

Do you have code that I can test out?

Yes (qualified) (alpha) (rough)

How are you setup to use GrovePi at the moment?

Are you using Dexter Industries own Raspbian for Robots or have you installed it into your own copy of Raspbian Stretch


chrowe
Scratcher
6 posts

Export/pipe data out of Scratch

Raspian for Robots, but I could set one with Raspian Stretch if that is easier.
SimpleScratch
Scratcher
500+ posts

Export/pipe data out of Scratch

Ok instead of using you as guinea pig for my Scratch 2 extension I've come up with a mod to exisiting Scratch 1.4 GrovePi Setup

This should work on your Raspbian for Robots as it doesn't have any specific Stretch aspects to it but make sure you make a copy of GrovePiScratch.py before starting
(I can't remember where it is on R4R as I haven't got a working copy of that one but it will be in a xxxxxxxxx/GrovePi/Software/Scratch folder
-my guess would be Dexter/GrovePi/Software/Scratch but I may be wrong)

Anyway - get yourself into a terminal and type
pip install paho-mqtt

to install python suport for mqtt

then replace GrovePiScratch.py with this version
#!/usr/bin/python
###############################################################################################################
# This library is for using the GrovePi with Scratch
# http://www.dexterindustries.com/GrovePi/
# History
# ------------------------------------------------
# Author Date Comments
# Karan 29 June 15 Initial Authoring
# John 22 Feb 16 Adding GrovePi Barometer
# Nicole Nov 16 Added Folder support for take_picture
# Nicole Nov 16 Added eSpeak Support
# Nicole 18 Nov 16 Adding PivotPi support
# SimonWalters 23 Feb 18 Correct error handling error
# SimonWalters 02 Mar 18 Added in MQTT support
'''
## License

The MIT License (MIT)

GrovePi for the Raspberry Pi: an open source platform for connecting Grove Sensors to the Raspberry Pi.
Copyright (C) 2016 Dexter Industries

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
'''
'''
#
# Based on the BrickPi Scratch Library written by Jaikrishna
#
# The Python program acts as the Bridge between Scratch & GrovePi and must be running for the Scratch program to run.
##############################################################################################################
'''
import scratch,sys,threading,math
import grovepi
import time
import os # to handle folder paths
import paho.mqtt.client as paho

def on_connect(client, userdata, flags, rc):
print("CONNACK received with code %d." % (rc))

def on_subscribe(client, userdata, mid, granted_qos):
print("Subscribed: "+str(mid)+" "+str(granted_qos))

def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
s.sensorupdate({msg.topic:str(msg.payload)})

def on_publish(client,userdata,result):
#create function for callback
print("data published \n")


client = paho.Client()
client.on_connect = on_connect
client.on_subscribe = on_subscribe
client.on_message = on_message
client.on_publish = on_publish



try:
sys.path.insert(0, '/home/pi/Dexter/PivotPi/Software/Scratch/')
import PivotPiScratch
pivotpi_available=True
except:
pivotpi_available=False

# Folders where pictures get saved
defaultCameraFolder="/home/pi/Desktop/"
cameraFolder = defaultCameraFolder


# Pi user ID Number - used for setting permissions
pi_user=1000
pi_group=1000

# used is GrovePi is actually connected - allows for testing without a GrovePi
en_grovepi=1
# print debugging statements
en_debug=1

try:
s = scratch.Scratch()
if s.connected:
print "GrovePi Scratch: Connected to Scratch successfully"
#else:
#sys.exit(0)
except scratch.ScratchError:
print "GrovePi Scratch: Scratch is either not opened or remote sensor connections aren't enabled"
#sys.exit(0)

class myThread (threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
while running:
time.sleep(.2) # sleep for 200 ms

thread1 = myThread(1, "Thread-1", 1) #Setup and start the thread
thread1.setDaemon(True)

analog_sensors=['analogRead','rotary','sound','light','moisture']
digitalInp=['button']
digitalOp=['led','relay']
pwm=['LEDPower','buzzer','analogWrite']

def match_sensors(msg,lst):
for i,e in enumerate(lst):
if msg[:len(e)].lower()==e.lower():
return i
return -1

try:
s.broadcast('READY')
except NameError:
print "GrovePi Scratch: Unable to Broadcast"


while True:
try:
m = s.receive()

while m==None or m[0] == 'sensor-update':
m = s.receive()

originalmsg = m[1]

# change input to all lowercase.
# Users can enter any which way they want
msg = originalmsg.lower()
if en_debug:
print "Rx:",originalmsg

if msg == 'SETUP'.lower() :
print "Setting up sensors done"
elif msg == 'START'.lower() :
running = True
if thread1.is_alive() == False:
thread1.start()
print "Service Started"

# ANALOG SENSORS
elif match_sensors(msg,analog_sensors) >=0:
if en_grovepi:
s_no=match_sensors(msg,analog_sensors)
sens=analog_sensors[s_no]
port=int(msg[len(sens):])
a_read=grovepi.analogRead(port)
s.sensorupdate({sens:a_read})

if en_debug:
print msg
print sens +'op:'+ str(a_read)

elif msg[:8].lower()=="setInput".lower():
if en_grovepi:
port=int(msg[8:])
grovepi.pinMode(port,"INPUT")
if en_debug:
print msg

elif msg[:9].lower()=="setOutput".lower():
if en_grovepi:
port=int(msg[9:])
grovepi.pinMode(port,"OUTPUT")
if en_debug:
print msg

elif msg[:11].lower()=="digitalRead".lower():
if en_grovepi:
port=int(msg[11:])
grovepi.pinMode(port,"INPUT")
d_read=grovepi.digitalRead(port)
s.sensorupdate({'digitalRead':d_read})
if en_debug:
print msg
print "Digital Reading: " + str(d_read)

elif match_sensors(msg,digitalInp) >=0:
if en_grovepi:
s_no=match_sensors(msg,digitalInp)
sens=digitalInp[s_no]
port=int(msg[len(sens):])
sens += str(port)
grovepi.pinMode(port,"INPUT")
d_read=grovepi.digitalRead(port)
s.sensorupdate({sens:d_read})
if en_debug:
print msg,
print sens +' output:'+ str(d_read)

elif msg[:16].lower()=="digitalWriteHigh".lower():
if en_grovepi:
port=int(msg[16:])
grovepi.pinMode(port,"OUTPUT")
grovepi.digitalWrite(port,1)
if en_debug:
print msg

elif msg[:15].lower()=="digitalWriteLow".lower():
if en_grovepi:
port=int(msg[15:])
grovepi.pinMode(port,"OUTPUT")
grovepi.digitalWrite(port,0)
if en_debug:
print msg

elif match_sensors(msg,pwm) >=0:
if en_grovepi:
s_no=match_sensors(msg,pwm)
sens=pwm[s_no]
l=len(sens)
port=int(msg[l:l+1])
power=int(msg[l+1:])
grovepi.pinMode(port,"OUTPUT")
grovepi.analogWrite(port,power)
if en_debug:
print msg

elif match_sensors(msg,digitalOp) >=0:
if en_grovepi:
s_no=match_sensors(msg,digitalOp)
sens=digitalOp[s_no]
l=len(sens)
port=int(msg[l:l+1])
state=msg[l+1:]
grovepi.pinMode(port,"OUTPUT")
if state=='on':
grovepi.digitalWrite(port,1)
else:
grovepi.digitalWrite(port,0)
if en_debug:
print msg

elif msg[:4].lower()=="temp".lower():
if en_grovepi:
port=int(msg[4:])
[temp,humidity] = grovepi.dht(port,0)
s.sensorupdate({'temp':temp})
if en_debug:
print msg
print "temp: ",temp

elif msg[:8].lower()=="humidity".lower():
if en_grovepi:
port=int(msg[8:])
[temp,humidity] = grovepi.dht(port,0)
s.sensorupdate({'humidity':humidity})
if en_debug:
print msg
print "humidity:",humidity

elif msg[:8].lower()=="distance".lower():
if en_grovepi:
port=int(msg[8:])
dist=grovepi.ultrasonicRead(port)
s.sensorupdate({'distance':dist})
if en_debug:
print msg
print "distance=",dist

elif msg[:3].lower()=="lcd".lower():
if en_grovepi:
if en_debug:
print msg[:3], msg[3:6], originalmsg[6:]
import grove_rgb_lcd
if msg[3:6].lower() == "col".lower(): #lower() added just for consistency. Not really needed
rgb = []
for i in range(0,6,2):
rgb.append(int(msg[6:][i:i+2],16)) # convert from one hex string to three ints
if en_debug:
print "colours are:",rgb[0],rgb[1],rgb[2]
grove_rgb_lcd.setRGB(rgb[0],rgb[1],rgb[2])
elif msg[3:6].lower() == "txt".lower():
txt = originalmsg[6:]
grove_rgb_lcd.setText_norefresh(txt)
else:
pass
if en_debug:
print msg

# elif msg[:10].lower()=="setOutput".lower():
# if en_grovepi:
# port=int(msg[10:])
# a_read=grovepi.analogRead(port)
# s.sensorupdate({'analogRead':a_read})
# if en_debug:
# print msg
# print "Analog Reading: " + str(a_read)
elif msg.lower()=="READ_IR".lower() or msg.lower()=="IR".lower():
print "READ_IR!"
if en_ir_sensor==0:
import lirc
sockid = lirc.init("keyes", blocking = False)
en_ir_sensor=1
try:
read_ir= lirc.nextcode() # press 1
if len(read_ir) !=0:
print read_ir[0]
except:
if en_debug:
e = sys.exc_info()[1]
print "Error reading IR sensor: " + str(read_ir)
if en_debug:
print "IR Recv Reading: " + str(read_ir)
if en_gpg:
if len(read_ir) !=0:
s.sensorupdate({'read_ir':read_ir[0]})
else:
s.sensorupdate({'read_ir':""})
# CREATE FOLDER TO SAVE PHOTOS IN

elif msg[:6].lower()=="FOLDER".lower():
print "Camera folder"
try:
cameraFolder=defaultCameraFolder+str(msg[6:])
if not os.path.exists(cameraFolder):
os.makedirs(cameraFolder)
os.chown(cameraFolder,pi_user,pi_group)
s.sensorupdate({"folder":"created"})
else:
s.sensorupdate({"folder":"set"})
except:
print "error with folder name"

elif msg.lower()=="TAKE_PICTURE".lower():
print "TAKE_PICTURE!"
try:
from subprocess import call
import datetime
newimage = "{}/img_{}.jpg".format(cameraFolder,str(datetime.datetime.now()).replace(" ","_",10).replace(":","_",10))
photo_cmd="raspistill -o {} -w 640 -h 480 -t 1".format(newimage)
print photo_cmd
call ([photo_cmd], shell=True)
os.chown(newimage,pi_user,pi_group)
print "Picture Taken"
except:
if en_debug:
e = sys.exc_info()[1]
print "Error taking picture",e
s.sensorupdate({'camera':"Error"})
s.sensorupdate({'camera':"Picture Taken"})

# Barometer code, pressure
elif msg[:9].lower()=="pressure".lower():
if en_grovepi:
# We import here to prevent errors thrown. If the import fails, you just get an error message instead of the communicator crashing.
# If user is using multiple sensors and using their own image which does not have the pythonpath set correctly then
# they'll just not get the output for 1 sensor, and the others will still keep working
from grove_i2c_barometic_sensor_BMP180 import BMP085 # Barometric pressure sensor.
bmp = BMP085(0x77, 1) #Initialize the pressure sensor (barometer)
press = bmp.readPressure()/100.0
s.sensorupdate({'pressure':press})
if en_debug:
print "Pressure: " + str(press)
if en_debug: # If Debug is enabled, print the value of msg.
print msg

elif (msg[:5].lower()=="SPEAK".lower()):
try:
if en_grovepi:
from subprocess import call
cmd_beg = "espeak -ven+f1 "
in_text = msg[len("SPEAK"):]
cmd_end = " 2>/dev/null"

call([cmd_beg+"\""+in_text+"\""+cmd_end], shell=True)
if en_debug:
print(msg)
except:
print("Issue with espeak")

# PIVOTPI
elif pivotpi_available==True and PivotPiScratch.isPivotPiMsg(msg):
pivotsensors = PivotPiScratch.handlePivotPi(msg)
# print "Back from PivotPi",pivotsensors
s.sensorupdate(pivotsensors)

elif (msg[:4].lower() == "mqtt"):
print "mqtt messge:",msg
splits = msg.lower().split(",")
print splits
if (splits[0][:11] == "mqttconnect"):
try:
client.disconnect()
client.loop_stop()
print "client stopped"
except:
print "no client running"
pass
print "carry on"
client.connect(splits[0][11:],1883)
client.loop_start()
elif (splits[0][:13] == "mqttsubscribe"):
client.subscribe(splits[0][13:])
elif (splits[0][:11] == "mqttpublish"):
client.publish(splits[0][11:],splits[1])
else:
if en_debug:
print "Ignoring: ",msg



except KeyboardInterrupt:
running= False
print "GrovePi Scratch: Disconnected from Scratch"
break
except (scratch.ScratchConnectionError,NameError) as e:
while True:
#thread1.join(0)
print "GrovePi Scratch: Scratch connection error, Retrying"
# print e
time.sleep(5)
try:
s = scratch.Scratch()
s.broadcast('READY')
print "GrovePi Scratch: Connected to Scratch successfully"
break;
except scratch.ScratchError:
print "GrovePi Scratch: Scratch is either not opened or remote sensor connections aren't enabled\n..............................\n"
except:
e = sys.exc_info()[0]
print "GrovePi Scratch: Error %s" % e

Then run your GrovePi Scratch like you normally do
Make sure a normal GrovePi broadcast still works ok
if so - try out these extra ones
broadcast mqttconnecttest.mosquitto.org
broadcast mqttsubscribecheerlights

if all goes well - you should see new sensor created called cheerlights

to publish to a topic
broadcast mqttpublishxxx,yyy
where xx is your topic and yyy is your message

Last edited by SimpleScratch (March 2, 2018 19:52:50)

myeducate
Scratcher
500+ posts

Export/pipe data out of Scratch

By the way guys, you can use the
[code][/code]
tag to put your code in.

INACTIVE
SPA Member and Assosiate - Creator and overlord of ScratchNetwork - 700+ Forum Posts - Web and Desktop Dev - Fluent in VB, PHP and HTML. I'm okay at CSS and Javascript but am still learning. Sig written in PHP using the picture libary. Firebase is fun.

SimpleScratch
Scratcher
500+ posts

Export/pipe data out of Scratch

Ta
Air_heads
Scratcher
79 posts

Export/pipe data out of Scratch

How do you implement python exports (such as profile dats ) into scratch? how to you do it? what do you use? please help.

Powered by DjangoBB