Compare commits
16 Commits
a47304a961
...
97c22977d0
Author | SHA1 | Date |
---|---|---|
Samuel Ortion | 97c22977d0 | |
Samuel Ortion | 7a175e7f0a | |
Samuel Ortion | 4f09a2dd4e | |
Samuel Ortion | 39233fe937 | |
Samuel Ortion | fba68c6c90 | |
Samuel Ortion | 0d696c8399 | |
Samuel Ortion | 30b1b44876 | |
Samuel Ortion | 089acdffd9 | |
Samuel Ortion | daffa3ff96 | |
Samuel Ortion | d8ca495471 | |
Samuel Ortion | 1654885838 | |
Samuel Ortion | 082823892c | |
Samuel Ortion | a557723b35 | |
Samuel Ortion | c1af47eb26 | |
Samuel Ortion | 048e93fef0 | |
Samuel Ortion | 9d0cb08792 |
|
@ -1,4 +1,7 @@
|
|||
var
|
||||
.venv
|
||||
.github
|
||||
.ideas
|
||||
/var
|
||||
/.venv
|
||||
/.github
|
||||
/.ideas
|
||||
/media
|
||||
/daemon/systemd
|
||||
/analyzer
|
||||
|
|
|
@ -0,0 +1,12 @@
|
|||
# BirdNET-Analyzer environment
|
||||
CUDA_VISIBLE_DEVICES=""
|
||||
|
||||
# BirdNET-stream environment
|
||||
DATABASE_USER="birdnet"
|
||||
DATABASE_PASSWORD="secret" # change this
|
||||
DATABASE_PORT="3306"
|
||||
DATABASE_ROOT_PASSWORD="secret" # change this
|
||||
|
||||
RECORDS_DIR="/media/data/birdnet/records"
|
||||
CHARTS_DIR="/media/data/birdnet/charts"
|
||||
SERVER_NAME="birdnet.local"
|
|
@ -1,7 +1,10 @@
|
|||
var/
|
||||
/.venv/
|
||||
|
||||
.env
|
||||
/.env
|
||||
/.env.local
|
||||
!.env.local.example
|
||||
!.env.example
|
||||
|
||||
species_list.txt
|
||||
|
||||
|
|
|
@ -1,6 +1,11 @@
|
|||
# Changelog
|
||||
## v0.0.1-alpha
|
||||
|
||||
## v0.0.1-rc
|
||||
- Add docker compose port
|
||||
- Improve install script
|
||||
- Add base uninstall script (need deeper work)
|
||||
- Add ttyd for systemd logging
|
||||
## v0.0.1-rc (2022-08-18)
|
||||
|
||||
- Integrate BirdNET-Analyzer as submodule
|
||||
- Add birdnet_recording service
|
||||
|
@ -11,4 +16,4 @@
|
|||
- Add /today/species and /today/{date}/species/{id} endpoints
|
||||
- Add records deletion button and /records/delete endpoint as well as bulk deletion (select all button on /today/species/{id} endpoint)
|
||||
- Add systemd status page /status
|
||||
- Add i18n for webapp (not species name), en|fr only for the moment
|
||||
- Add i18n for webapp (not species name), en|fr only for the moment
|
20
INSTALL.md
20
INSTALL.md
|
@ -79,17 +79,23 @@ sudo systemctl enable --now birdnet_recording.service birdnet_analyzis.service b
|
|||
#### Check if services are working
|
||||
|
||||
```bash
|
||||
# Sercices status
|
||||
sudo systemctl status birdnet_recording.service birdnet_analyzis.service
|
||||
# Timers status
|
||||
sudo systemctl status birdnet_miner.timer
|
||||
# Sercices and timers status
|
||||
sudo systemctl status birdnet_\*
|
||||
```
|
||||
|
||||
```bash
|
||||
# BirdNET-stream logs
|
||||
sudo journalctl -feu {birdnet_recording,birdnet_analyzis}.service
|
||||
sudo journalctl -feu birdnet_\*
|
||||
```
|
||||
|
||||
#### Enable `loginctl-linger` for the user that runs the servuces
|
||||
|
||||
Running:
|
||||
```bash
|
||||
loginctl enable-linger
|
||||
```
|
||||
This allows to use `/run/user/1000/pulse` to record audio using PulseAudio in birdnet_recording.sh.
|
||||
|
||||
## Setup BirdNET-stream symfony webapp
|
||||
|
||||
### Install php 8.1
|
||||
|
@ -123,7 +129,7 @@ sudo mv /composer.phar /usr/local/bin/composer
|
|||
|
||||
```bash
|
||||
cd www
|
||||
composer install
|
||||
composer install --no-dev --prefer-dist --optimize-autoloader
|
||||
```
|
||||
|
||||
### Install nodejs and npm
|
||||
|
@ -141,7 +147,7 @@ nvm use 16
|
|||
```
|
||||
|
||||
```bash
|
||||
sudo dnf install npm
|
||||
sudo apt-get install npm
|
||||
```
|
||||
|
||||
```bash
|
||||
|
|
36
README.md
36
README.md
|
@ -8,7 +8,7 @@
|
|||
|
||||
## Introduction
|
||||
|
||||
BirdNET-stream records sound 24/7 on any Linux computer with a microphone and analyze it using BirdNET algorithm by [**@kahst**](https://github.com/kahst).
|
||||
BirdNET-stream records sound 24/7 on any Linux computer with a microphone and analyze it using BirdNET algorithm by [**@kahst**](https://github.com/kahst).
|
||||
|
||||
Bird contacts are stored in a database and are made accessible in a webapp.
|
||||
|
||||
|
@ -28,19 +28,45 @@ It should work on a Raspberry Pi (or other Single Board Computer) with a USB mic
|
|||
|
||||
## Installation
|
||||
|
||||
On debian based system, you can install BirdNET-stream with the following command:
|
||||
> **Warning** BirdNET-stream is in early development, and may not work properly...
|
||||
|
||||
<!-- On debian based system, you can install BirdNET-stream with the following command:
|
||||
|
||||
```bash
|
||||
curl -sL https://raw.githubusercontent.com/UncleSamulus/BirdNET-stream/main/install.sh | bash
|
||||
``` -->
|
||||
|
||||
On debian based systems (tested on Debian Bullseye), the following command should allow you to install the base components without too much trouble:
|
||||
|
||||
```bash
|
||||
# Change to your installation directory here, /home/$USER/Documents/BirdNET-stream for instance, or /opt/birdnet-stream, or whatever
|
||||
cd /path/to/installation/directory
|
||||
# Download installation script
|
||||
curl -0 https://raw.githubusercontent.com/UncleSamulus/BirdNET-stream/main/install.sh
|
||||
# Run installation script:
|
||||
chmod +x ./install.sh
|
||||
./install.sh
|
||||
```
|
||||
|
||||
For finer control, or to adapt to your system, you can follow the instructions in the [INSTALL.md](./INSTALL.md) file.
|
||||
I recommend to add `DEBUG=1` before this command to see the installation steps:
|
||||
```bash
|
||||
DEBUG=1 ./install.sh
|
||||
```
|
||||
|
||||
To install from a specific git branch, add `BRANCH=<branch>` before the command, for instance:
|
||||
|
||||
```bash
|
||||
BRANCH=dev DEBUG=1 ./install.sh
|
||||
```
|
||||
|
||||
For finer control, or to adapt to your system, you can follow the instructions in the [INSTALL.md](./INSTALL.md) file (it may unfortunatly not be accurate for your system).
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
- BirdNET-stream web application can be accessed on any web browser at [https://birdnet.home](https://birdnet.home), from your local network, or at any other hostname you set in nginx configuration, if your public IP is accessible from the internet.
|
||||
|
||||
- See the species detected
|
||||
- See the species detected
|
||||
|
||||
## Acknoledgments
|
||||
|
||||
|
@ -49,4 +75,4 @@ For finer control, or to adapt to your system, you can follow the instructions i
|
|||
|
||||
## License
|
||||
|
||||
BirdNET-stream is licensed under the GNU General Public License v3.0, see [./LICENSE](./LICENSE) for more details.
|
||||
BirdNET-stream is licensed under the GNU General Public License v3.0, see [./LICENSE](./LICENSE) for more details.
|
||||
|
|
1
TODO
1
TODO
|
@ -1,5 +1,4 @@
|
|||
- Fix service manager
|
||||
- Add docker support
|
||||
- Species i18n
|
||||
- File purge policy
|
||||
- Add and test RTSP support
|
|
@ -19,6 +19,4 @@ PYTHON_VENV="./.venv/birdnet-stream"
|
|||
WORKDIR="/home/$USER/BirdNET-stream"
|
||||
# Database location
|
||||
DATABASE="./var/db.sqlite"
|
||||
|
||||
DAEMON_USER="birdnet"
|
||||
DAEMON_PASSWORD="secret"
|
||||
# DATABASE="mysql://birdnet:secret@localhost:3306/birdnet_observations" # uncomment and change 'secret' if you want to use a mariadb (mysql) database instea of sqlite
|
||||
|
|
|
@ -3,9 +3,7 @@ set -e
|
|||
|
||||
DEBUG=${DEBUG:-1}
|
||||
debug() {
|
||||
if [ $DEBUG -eq 1 ]; then
|
||||
echo "$1"
|
||||
fi
|
||||
[[ $DEBUG -eq 1 ]] && echo "$@"
|
||||
}
|
||||
|
||||
config_filepath="./config/birdnet.conf"
|
||||
|
@ -64,7 +62,9 @@ check_prerequisites() {
|
|||
|
||||
# Get array of audio chunks to be processed
|
||||
get_chunk_list() {
|
||||
find "${CHUNK_FOLDER}/in" -type f -name '*.wav' -exec basename {} \; ! -size 0 | sort
|
||||
chunk_list=($(ls ${CHUNK_FOLDER}/in))
|
||||
echo "${chunk_list}"
|
||||
# find "${CHUNK_FOLDER}/in" -type f -name '*.wav' -exec basename {} \; ! -size 0 | sort
|
||||
}
|
||||
|
||||
# Perform audio chunk analysis on one chunk
|
||||
|
@ -75,13 +75,22 @@ analyze_chunk() {
|
|||
mkdir -p "$output_dir"
|
||||
date=$(echo $chunk_name | cut -d'_' -f2)
|
||||
week=$(./daemon/weekof.sh $date)
|
||||
$PYTHON_EXECUTABLE ./analyzer/analyze.py --i $chunk_path --o "$output_dir/model.out.csv" --lat $LATITUDE --lon $LONGITUDE --week $week --min_conf $CONFIDENCE --threads 4 --rtype csv
|
||||
if [[ ! -z "${THREADS}" ]]; then
|
||||
threads="--threads ${THREADS}"
|
||||
else
|
||||
threads=""
|
||||
fi
|
||||
$PYTHON_EXECUTABLE ./analyzer/analyze.py --i $chunk_path --o "$output_dir/model.out.csv" --lat $LATITUDE --lon $LONGITUDE --week $week --min_conf $CONFIDENCE $threads --rtype csv
|
||||
debug "Model output written to $output_dir/model.out.csv"
|
||||
bash ./daemon/birdnet_output_to_sql.sh "$output_dir/model.out.csv"
|
||||
debug "Dumped to SQL database"
|
||||
}
|
||||
|
||||
# Perform audio chunk analysis on all recorded chunks
|
||||
analyze_chunks() {
|
||||
for chunk_name in $(get_chunk_list); do
|
||||
local chunks
|
||||
chunks="${1}"
|
||||
for chunk_name in "${chunks}"; do
|
||||
if [[ -f "${CHUNK_FOLDER}/out/$chunk_name.d/model.out.csv" ]]; then
|
||||
debug "Skipping $chunk_name, as it has already been analyzed"
|
||||
else
|
||||
|
@ -98,4 +107,4 @@ check_prerequisites
|
|||
chunks=$(get_chunk_list)
|
||||
|
||||
# Analyze all chunks in working directory
|
||||
analyze_chunks $chunks
|
||||
analyze_chunks "$chunks"
|
|
@ -1,15 +1,13 @@
|
|||
#! /usr/bin/env bash
|
||||
# Extract observations from a model output folder
|
||||
# Extract observations from a model output file into SQL database
|
||||
#
|
||||
|
||||
DEBUG=${DEBUG:-1}
|
||||
set -e
|
||||
# set -x
|
||||
|
||||
DEBUG=${DEBUG:-1}
|
||||
debug() {
|
||||
if [ $DEBUG -eq 1 ]; then
|
||||
echo "$1"
|
||||
fi
|
||||
[[ $DEBUG -eq 1 ]] && echo "$@"
|
||||
}
|
||||
|
||||
# Load bash library to deal with BirdNET-stream database
|
||||
|
@ -18,16 +16,6 @@ source ./daemon/database/scripts/database.sh
|
|||
# Load config
|
||||
source ./config/birdnet.conf
|
||||
# Check config
|
||||
if [[ -z ${CHUNK_FOLDER} ]]; then
|
||||
echo "CHUNK_FOLDER is not set"
|
||||
exit 1
|
||||
else
|
||||
if [[ ! -d ${CHUNK_FOLDER}/out ]]; then
|
||||
echo "CHUNK_FOLDER does not exist: ${CHUNK_FOLDER}/out"
|
||||
echo "Cannot extract observations."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ -z ${LATITUDE} ]]; then
|
||||
echo "LATITUDE is not set"
|
||||
|
@ -39,10 +27,6 @@ if [[ -z ${LONGITUDE} ]]; then
|
|||
exit 1
|
||||
fi
|
||||
|
||||
model_outputs() {
|
||||
ls ${CHUNK_FOLDER}/out/*/model.out.csv
|
||||
}
|
||||
|
||||
source_wav() {
|
||||
model_output_path=$1
|
||||
model_output_dir=$(dirname $model_output_path)
|
||||
|
@ -107,13 +91,6 @@ save_observations() {
|
|||
done
|
||||
}
|
||||
|
||||
main() {
|
||||
# # Remove all junk observations
|
||||
# ./daemon/birdnet_clean.sh
|
||||
# Get model outputs
|
||||
for model_output in $(model_outputs); do
|
||||
save_observations $model_output
|
||||
done
|
||||
}
|
||||
model_output_path="$1"
|
||||
|
||||
main
|
||||
save_observations $model_output_path
|
|
@ -3,6 +3,7 @@
|
|||
DEBUG=${DEBUG:-1}
|
||||
|
||||
export PULSE_RUNTIME_PATH="/run/user/$(id -u)/pulse/"
|
||||
FFMPEG_OPTIONS="-nostdin -hide_banner -loglevel error -nostats -vn -acodec pcm_s16le -ac 1 -ar 48000"
|
||||
|
||||
debug() {
|
||||
if [ $DEBUG -eq 1 ]; then
|
||||
|
@ -27,20 +28,25 @@ record_loop() {
|
|||
done
|
||||
}
|
||||
|
||||
FFMPEG_OPTIONS="-nostdin -hide_banner -loglevel error -nostats -vn -acodec pcm_s16le -ac 1 -ar 48000 "
|
||||
|
||||
record_stream() {
|
||||
local STREAM=$1
|
||||
local DURATION=$2
|
||||
local debug "Recording from $STREAM for $DURATION seconds"
|
||||
ffmpeg $FFMPEG_OPTIONS -f -i ${DEVICE} -t ${DURATION} file:${CHUNK_FOLDER}/in/birdnet_$(date "+%Y%m%d_%H%M%S").wav
|
||||
ffmpeg $FFMPEG_OPTIONS -i ${STREAM} -t ${DURATION} file:${CHUNK_FOLDER}/in/birdnet_$(date "+%Y%m%d_%H%M%S").wav
|
||||
}
|
||||
|
||||
record_device() {
|
||||
DEVICE=$1
|
||||
DURATION=$2
|
||||
debug "Recording from $DEVICE for $DURATION seconds"
|
||||
ffmpeg $FFMPEG_OPTIONS -f pulse -i ${DEVICE} -t ${DURATION} -af "volume=$RECORDING_AMPLIFY" file:${CHUNK_FOLDER}/in/birdnet_$(date "+%Y%m%d_%H%M%S").wav
|
||||
local ffmpeg_input
|
||||
if [[ "$AUDIO_USE_PULSE" = "true" ]]; then
|
||||
ffmpeg_input="-f pulse -i ${DEVICE}"
|
||||
else
|
||||
ffmpeg_input="-f alsa -i ${DEVICE}"
|
||||
fi
|
||||
ffmpeg $FFMPEG_OPTIONS $ffmpeg_input -t ${DURATION} -af "volume=$RECORDING_AMPLIFY" file:${CHUNK_FOLDER}/in/birdnet_$(date "+%Y%m%d_%H%M%S").wav
|
||||
}
|
||||
|
||||
config_filepath="./config/birdnet.conf"
|
||||
|
|
|
@ -17,5 +17,16 @@ if [ -z "$DATABASE" ]; then
|
|||
DATABASE="./var/db.sqlite"
|
||||
fi
|
||||
|
||||
# Create database according to schema in structure.sql
|
||||
sqlite3 "$DATABASE" < ./daemon/database/structure.sql
|
||||
if [[ $DATABASE = "mysql://"* ]]; then
|
||||
# Split mysql uri into user, password, host, port, and database
|
||||
MYSQL_ADDRESS=$(echo "$DATABASE" | sed 's/mysql:\/\///g')
|
||||
MYSQL_CREDENTIALS=$(echo "$MYSQL_ADDRESS" | cut -d@ -f1)
|
||||
MYSQL_USER=$(echo "$MYSQL_CREDENTIALS" | cut -d: -f1)
|
||||
MYSQL_PASSWORD=$(echo "$MYSQL_CREDENTIALS" | cut -d: -f2)
|
||||
MYSQL_HOST=$(echo "$MYSQL_ADDRESS" | cut -d@ -f2 | cut -d: -f1)
|
||||
MYSQL_PORT=$(echo "$MYSQL_ADDRESS" | cut -d@ -f2 | cut -d: -f2 | cut -d/ -f1)
|
||||
MYSQL_DATABASE=$(echo "$MYSQL_ADDRESS" | cut -d/ -f2)
|
||||
mysql -u$MYSQL_USER -p$MYSQL_PASSWORD -h$MYSQL_HOST -P$MYSQL_PORT -D$MYSQL_DATABASE < ./daemon/database/structure-mysql.sql
|
||||
else
|
||||
sqlite3 $DATABASE < ./daemon/database/structure-sqlite.sql
|
||||
fi
|
||||
|
|
|
@ -8,10 +8,30 @@ source ./config/birdnet.conf
|
|||
# Create database in case it was not created yet
|
||||
./daemon/database/scripts/create.sh
|
||||
|
||||
DATABASE=${DATABASE:-"./var/db.sqlite"}
|
||||
# Check if database location is specified
|
||||
if [ -z "$DATABASE" ]; then
|
||||
echo "DATABASE location not specified"
|
||||
echo "Defaults to ./var/db.sqlite"
|
||||
DATABASE="./var/db.sqlite"
|
||||
fi
|
||||
|
||||
query() {
|
||||
sqlite3 -cmd ".timeout 1000" $DATABASE "$1"
|
||||
local stmt
|
||||
stmt="$1"
|
||||
if [[ $DATABASE = "mysql://"* ]]; then
|
||||
# Split mysql uri into user, password, host, port, and database
|
||||
MYSQL_ADDRESS=$(echo "$DATABASE" | sed 's/mysql:\/\///g')
|
||||
MYSQL_CREDENTIALS=$(echo "$MYSQL_ADDRESS" | cut -d@ -f1)
|
||||
MYSQL_USER=$(echo "$MYSQL_CREDENTIALS" | cut -d: -f1)
|
||||
MYSQL_PASSWORD=$(echo "$MYSQL_CREDENTIALS" | cut -d: -f2)
|
||||
MYSQL_HOST=$(echo "$MYSQL_ADDRESS" | cut -d@ -f2 | cut -d: -f1)
|
||||
MYSQL_PORT=$(echo "$MYSQL_ADDRESS" | cut -d@ -f2 | cut -d: -f2 | cut -d/ -f1)
|
||||
MYSQL_DATABASE=$(echo "$MYSQL_ADDRESS" | cut -d/ -f2)
|
||||
mysql -u$MYSQL_USER -p$MYSQL_PASSWORD -h$MYSQL_HOST -P$MYSQL_PORT -D$MYSQL_DATABASE -e "$stmt"
|
||||
else
|
||||
sqlite3 -cmd ".timeout 1000" "$DATABASE" "$stmt"
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
get_location_id() {
|
||||
|
@ -37,4 +57,4 @@ insert_observation() {
|
|||
# Check if the observation already exists in the database
|
||||
observation_exists() {
|
||||
query "SELECT EXISTS(SELECT observation_id FROM observation WHERE audio_file='$1' AND start='$2' AND end='$3' AND taxon_id='$4' AND location_id='$5')"
|
||||
}
|
||||
}
|
||||
|
|
|
@ -0,0 +1,31 @@
|
|||
/** Database structure for BirdNET-stream SQLite*/
|
||||
|
||||
/** Taxon table */
|
||||
CREATE TABLE IF NOT EXISTS taxon (
|
||||
taxon_id INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT,
|
||||
scientific_name TEXT NOT NULL,
|
||||
common_name TEXT NOT NULL
|
||||
);
|
||||
|
||||
/** Location table */
|
||||
CREATE TABLE IF NOT EXISTS location (
|
||||
location_id INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT,
|
||||
latitude REAL NOT NULL,
|
||||
longitude REAL NOT NULL
|
||||
);
|
||||
|
||||
/** Observation table */
|
||||
CREATE TABLE IF NOT EXISTS observation (
|
||||
`observation_id` INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT,
|
||||
`audio_file` TEXT NOT NULL,
|
||||
`start` REAL NOT NULL,
|
||||
`end` REAL NOT NULL,
|
||||
`taxon_id` INTEGER NOT NULL,
|
||||
`location_id` INTEGER NOT NULL,
|
||||
`date` TEXT NOT NULL,
|
||||
`notes` TEXT,
|
||||
`confidence` REAL NOT NULL,
|
||||
`verified` BOOLEAN DEFAULT 0 CHECK (`verified` IN (0, 1)),
|
||||
FOREIGN KEY(taxon_id) REFERENCES taxon(taxon_id),
|
||||
FOREIGN KEY(location_id) REFERENCES location(location_id)
|
||||
);
|
|
@ -2,21 +2,21 @@
|
|||
|
||||
/** Taxon table */
|
||||
CREATE TABLE IF NOT EXISTS taxon (
|
||||
taxon_id INTEGER PRIMARY KEY,
|
||||
taxon_id INTEGER PRIMARY KEY NOT NULL,
|
||||
scientific_name TEXT NOT NULL,
|
||||
common_name TEXT NOT NULL
|
||||
);
|
||||
|
||||
/** Location table */
|
||||
CREATE TABLE IF NOT EXISTS location (
|
||||
location_id INTEGER PRIMARY KEY,
|
||||
location_id INTEGER PRIMARY KEY NOT NULL,
|
||||
latitude REAL NOT NULL,
|
||||
longitude REAL NOT NULL
|
||||
);
|
||||
|
||||
/** Observation table */
|
||||
CREATE TABLE IF NOT EXISTS observation (
|
||||
`observation_id` INTEGER PRIMARY KEY,
|
||||
`observation_id` INTEGER PRIMARY KEY NOT NULL,
|
||||
`audio_file` TEXT NOT NULL,
|
||||
`start` REAL NOT NULL,
|
||||
`end` REAL NOT NULL,
|
||||
|
@ -25,7 +25,7 @@ CREATE TABLE IF NOT EXISTS observation (
|
|||
`date` TEXT NOT NULL,
|
||||
`notes` TEXT,
|
||||
`confidence` REAL NOT NULL,
|
||||
`verified` BOOLEAN NOT NULL CHECK (`verified` IN (0, 1)) DEFAULT 0,
|
||||
`verified` BOOLEAN DEFAULT 0 CHECK (`verified` IN (0, 1)),
|
||||
FOREIGN KEY(taxon_id) REFERENCES taxon(taxon_id),
|
||||
FOREIGN KEY(location_id) REFERENCES location(location_id)
|
||||
);
|
|
@ -6,90 +6,119 @@ import matplotlib.pyplot as plt
|
|||
from matplotlib.colors import LogNorm
|
||||
import seaborn as sns
|
||||
from datetime import datetime
|
||||
import os
|
||||
import glob
|
||||
|
||||
CONFIG = {
|
||||
"readings": 10,
|
||||
"palette": "Greens",
|
||||
"db": "./var/db.sqlite",
|
||||
"date": datetime.now().strftime("%Y-%m-%d")
|
||||
# "date": "2022-08-15"
|
||||
"date": datetime.now().strftime("%Y-%m-%d"),
|
||||
"charts_dir": "./var/charts"
|
||||
}
|
||||
|
||||
db = sqlite3.connect(CONFIG['db'])
|
||||
db = None
|
||||
|
||||
df = pd.read_sql_query("""SELECT common_name, date, location_id, confidence
|
||||
FROM observation
|
||||
INNER JOIN taxon
|
||||
ON observation.taxon_id = taxon.taxon_id""", db)
|
||||
df['date'] = pd.to_datetime(df['date'])
|
||||
df['hour'] = df['date'].dt.hour
|
||||
df['date'] = df['date'].dt.date
|
||||
df['date'] = df['date'].astype(str)
|
||||
df_on_date = df[df['date'] == CONFIG['date']]
|
||||
|
||||
top_on_date = (df_on_date['common_name'].value_counts()[:CONFIG['readings']])
|
||||
if top_on_date.empty:
|
||||
print("No observations on {}".format(CONFIG['date']))
|
||||
exit()
|
||||
def get_database():
|
||||
global db
|
||||
if db is None:
|
||||
db = sqlite3.connect(CONFIG["db"])
|
||||
return db
|
||||
|
||||
df_top_on_date = df_on_date[df_on_date['common_name'].isin(top_on_date.index)]
|
||||
def chart(date):
|
||||
db = get_database()
|
||||
df = pd.read_sql_query(f"""SELECT common_name, date, location_id, confidence
|
||||
FROM observation
|
||||
INNER JOIN taxon
|
||||
ON observation.taxon_id = taxon.taxon_id
|
||||
WHERE STRFTIME("%Y-%m-%d", `date`) = '{date}'""", db)
|
||||
df['date'] = pd.to_datetime(df['date'])
|
||||
df['hour'] = df['date'].dt.hour
|
||||
df['date'] = df['date'].dt.date
|
||||
df['date'] = df['date'].astype(str)
|
||||
df_on_date = df[df['date'] == date]
|
||||
|
||||
top_on_date = (df_on_date['common_name'].value_counts()[:CONFIG['readings']])
|
||||
if top_on_date.empty:
|
||||
print("No observations on {}".format(date))
|
||||
return
|
||||
else:
|
||||
print(f"Found observations on {date}")
|
||||
|
||||
df_top_on_date = df_on_date[df_on_date['common_name'].isin(top_on_date.index)]
|
||||
|
||||
# Create a figure with 2 subplots
|
||||
fig, axs = plt.subplots(1, 2, figsize=(20, 5), gridspec_kw=dict(
|
||||
width_ratios=[2, 6]))
|
||||
plt.subplots_adjust(left=None, bottom=None, right=None,
|
||||
top=None, wspace=0, hspace=0)
|
||||
# Create a figure with 2 subplots
|
||||
fig, axs = plt.subplots(1, 2, figsize=(20, 5), gridspec_kw=dict(
|
||||
width_ratios=[2, 6]))
|
||||
plt.subplots_adjust(left=None, bottom=None, right=None,
|
||||
top=None, wspace=0, hspace=0)
|
||||
|
||||
# Get species frequencies
|
||||
frequencies_order = pd.value_counts(df_top_on_date['common_name']).iloc[:CONFIG['readings']].index
|
||||
# Get min max confidences
|
||||
confidence_minmax = df_top_on_date.groupby('common_name')['confidence'].max()
|
||||
confidence_minmax = confidence_minmax.reindex(frequencies_order)
|
||||
# Norm values for color palette
|
||||
norm = plt.Normalize(confidence_minmax.values.min(),
|
||||
confidence_minmax.values.max())
|
||||
# Get species frequencies
|
||||
frequencies_order = pd.value_counts(df_top_on_date['common_name']).iloc[:CONFIG['readings']].index
|
||||
# Get min max confidences
|
||||
confidence_minmax = df_top_on_date.groupby('common_name')['confidence'].max()
|
||||
confidence_minmax = confidence_minmax.reindex(frequencies_order)
|
||||
# Norm values for color palette
|
||||
norm = plt.Normalize(confidence_minmax.values.min(),
|
||||
confidence_minmax.values.max())
|
||||
|
||||
colors = plt.cm.Greens(norm(confidence_minmax))
|
||||
plot = sns.countplot(y='common_name', data=df_top_on_date, palette=colors, order=frequencies_order, ax=axs[0])
|
||||
colors = plt.cm.Greens(norm(confidence_minmax))
|
||||
plot = sns.countplot(y='common_name', data=df_top_on_date, palette=colors, order=frequencies_order, ax=axs[0])
|
||||
|
||||
plot.set(ylabel=None)
|
||||
plot.set(xlabel="Detections")
|
||||
plot.set(ylabel=None)
|
||||
plot.set(xlabel="Detections")
|
||||
|
||||
heat = pd.crosstab(df_top_on_date['common_name'], df_top_on_date['hour'])
|
||||
# Order heatmap Birds by frequency of occurrance
|
||||
heat.index = pd.CategoricalIndex(heat.index, categories=frequencies_order)
|
||||
heat.sort_index(level=0, inplace=True)
|
||||
heat = pd.crosstab(df_top_on_date['common_name'], df_top_on_date['hour'])
|
||||
# Order heatmap Birds by frequency of occurrance
|
||||
heat.index = pd.CategoricalIndex(heat.index, categories=frequencies_order)
|
||||
heat.sort_index(level=0, inplace=True)
|
||||
|
||||
hours_in_day = pd.Series(data=range(0, 24))
|
||||
heat_frame = pd.DataFrame(data=0, index=heat.index, columns=hours_in_day)
|
||||
heat = (heat + heat_frame).fillna(0)
|
||||
hours_in_day = pd.Series(data=range(0, 24))
|
||||
heat_frame = pd.DataFrame(data=0, index=heat.index, columns=hours_in_day)
|
||||
heat = (heat + heat_frame).fillna(0)
|
||||
|
||||
# Generate heatmap plot
|
||||
plot = sns.heatmap(
|
||||
heat,
|
||||
norm=LogNorm(),
|
||||
annot=True,
|
||||
annot_kws={
|
||||
"fontsize": 7
|
||||
},
|
||||
fmt="g",
|
||||
cmap=CONFIG['palette'],
|
||||
square=False,
|
||||
cbar=False,
|
||||
linewidth=0.5,
|
||||
linecolor="Grey",
|
||||
ax=axs[1],
|
||||
yticklabels=False)
|
||||
plot.set_xticklabels(plot.get_xticklabels(), rotation=0, size=7)
|
||||
# Generate heatmap plot
|
||||
plot = sns.heatmap(
|
||||
heat,
|
||||
norm=LogNorm(),
|
||||
annot=True,
|
||||
annot_kws={
|
||||
"fontsize": 7
|
||||
},
|
||||
fmt="g",
|
||||
cmap=CONFIG['palette'],
|
||||
square=False,
|
||||
cbar=False,
|
||||
linewidth=0.5,
|
||||
linecolor="Grey",
|
||||
ax=axs[1],
|
||||
yticklabels=False)
|
||||
plot.set_xticklabels(plot.get_xticklabels(), rotation=0, size=7)
|
||||
|
||||
for _, spine in plot.spines.items():
|
||||
spine.set_visible(True)
|
||||
for _, spine in plot.spines.items():
|
||||
spine.set_visible(True)
|
||||
|
||||
plot.set(ylabel=None)
|
||||
plot.set(xlabel="Hour of day")
|
||||
plt.suptitle(f"Top {CONFIG['readings']} species on {CONFIG['date']}", fontsize=14)
|
||||
plt.text(15, 11, f"(Updated on {datetime.now().strftime('%Y/%m-%d %H:%M')})")
|
||||
plt.savefig(f"./var/charts/chart_{CONFIG['date']}.png", dpi=300)
|
||||
plt.close()
|
||||
plot.set(ylabel=None)
|
||||
plot.set(xlabel="Hour of day")
|
||||
plt.suptitle(f"Top {CONFIG['readings']} species on {date}", fontsize=14)
|
||||
plt.text(15, 11, f"(Updated on {datetime.now().strftime('%Y/%m-%d %H:%M')})")
|
||||
plt.savefig(f"./var/charts/chart_{date}.png", dpi=300)
|
||||
print(f"Plot for {date} saved.")
|
||||
plt.close()
|
||||
|
||||
db.close()
|
||||
def main():
|
||||
done_charts = glob.glob(f"{CONFIG['charts_dir']}/*.png")
|
||||
last_modified = max(done_charts, key=os.path.getctime)
|
||||
last_modified_date = last_modified.split("_")[-1].split(".")[0]
|
||||
missing_dates = pd.date_range(start=last_modified_date, end=CONFIG['date'], freq='D')
|
||||
print(missing_dates)
|
||||
for missing_date in missing_dates:
|
||||
date = missing_date.strftime("%Y-%m-%d")
|
||||
chart(date)
|
||||
chart(CONFIG['date'])
|
||||
if db is not None:
|
||||
db.close()
|
||||
print("Done.")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
|
@ -1,13 +0,0 @@
|
|||
[Unit]
|
||||
Description=BirdNET-stream miner service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=<USER>
|
||||
Group=<GROUP>
|
||||
WorkingDirectory=<DIR>
|
||||
ExecStart=bash ./daemon/birdnet_miner.sh
|
||||
RemainAfterExit=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
|
@ -1,9 +0,0 @@
|
|||
[Unit]
|
||||
Description=BirdNET-stream miner Timer
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*:0/15
|
||||
Unit=birdnet_miner.service
|
||||
|
||||
[Install]
|
||||
WantedBy=basic.target
|
|
@ -1,36 +1,119 @@
|
|||
version: '3.8'
|
||||
|
||||
networks:
|
||||
birdnet_network:
|
||||
version: '3.9'
|
||||
|
||||
services:
|
||||
# database:
|
||||
# container_name: birdnet_database
|
||||
# image:
|
||||
# recording:
|
||||
# container_name: birdnet_recording
|
||||
# build:
|
||||
# context: .
|
||||
# dockerfile: ./docker/recording/Dockerfile
|
||||
# restart: unless-stopped
|
||||
# environment:
|
||||
# - CHUNK_FOLDER=${CHUNK_FOLDER:-/media/birdnet/records}
|
||||
# volumes:
|
||||
# - ${RECORDS_DIR:-/media/birdnet/records}:${RECORS_FOLDER:-/media/birdnet/records}
|
||||
# # Allow container to access to the hosts microphone
|
||||
# devices:
|
||||
# - /dev/snd:/dev/snd
|
||||
# analyzer:
|
||||
# container_name: birdnet_analyzer
|
||||
# build:
|
||||
# context: ./analyzer/
|
||||
# dockerfile: ./Dockerfile
|
||||
|
||||
php:
|
||||
container_name: birdnet_php
|
||||
image: php:8.1-fpm
|
||||
db:
|
||||
container_name: birdnet_database
|
||||
image: mariadb:latest
|
||||
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
|
||||
ports:
|
||||
- "${PHP_FPM_PORT:-9001}:9000"
|
||||
- ${DATABASE_PORT:-3306}:3306
|
||||
networks:
|
||||
- birdnet_network
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${DATABASE_ROOT_PASSWORD:-secret}'
|
||||
MYSQL_USER: ${DATABASE_USER:-birdnet}
|
||||
MYSQL_PASSWORD: ${DATABASE_PASSWORD:-secret}
|
||||
volumes:
|
||||
- ./docker/database/init:/docker-entrypoint-initdb.d
|
||||
restart: unless-stopped
|
||||
|
||||
php-fpm:
|
||||
container_name: birdnet_php-fpm
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./docker/php-fpm/Dockerfile
|
||||
ports:
|
||||
- '${PHP_FPM_PORT:-9000}:9000'
|
||||
networks:
|
||||
- birdnet_network
|
||||
environment:
|
||||
- APP_ENV=${APP_ENV:-prod}
|
||||
- APP_DEBUG=${APP_DEBUG:-true}
|
||||
- DATABASE_DEFAULT_URL=mysql://${DATABASE_USER:-birdnet}:${DATABASE_PASSWORD:-secret}@${DATABASE_HOST:-birdnet_database}:${DATABASE_PORT:-3306}/birdnet_default
|
||||
- DATABASE_OBSERVATIONS_URL=mysql://${DATABASE_USER:-birdnet}:${DATABASE_PASSWORD:-secret}@${DATABASE_HOST:-birdnet_database}:${DATABASE_PORT:-3306}/birdnet_observations
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- birdnet_app:${PROJECT_ROOT:-/opt/birdnet}
|
||||
|
||||
symfony:
|
||||
container_name: birdnet_symfony
|
||||
networks:
|
||||
- birdnet_network
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./docker/symfony/Dockerfile
|
||||
args:
|
||||
- DATABASE_DEFAULT_URL=mysql://${DATABASE_USER:-birdnet}:${DATABASE_PASSWORD:-secret}@${DATABASE_HOST:-birdnet_database}:${DATABASE_PORT:-3306}/birdnet_default
|
||||
- DATABASE_OBSERVATIONS_URL=mysql://${DATABASE_USER:-birdnet}:${DATABASE_PASSWORD:-secret}@${DATABASE_HOST:-birdnet_database}:${DATABASE_PORT:-3306}/birdnet_observations
|
||||
- RECORDS_DIR=/media/birdnet/records
|
||||
- CHARTS_DIR=/media/birdnet/charts
|
||||
restart: on-failure
|
||||
volumes:
|
||||
- birdnet_app:${PROJECT_ROOT:-/opt/birdnet}
|
||||
- birdnet_records:${RECORDS_DIR:-/media/birdnet/records}
|
||||
depends_on:
|
||||
- db
|
||||
|
||||
nginx:
|
||||
container_name: birdnet_nginx
|
||||
hostname: ${SERVER_NAME:-birdnet.local}
|
||||
build:
|
||||
context: ./docker/
|
||||
environment:
|
||||
SERVER_NAME: ${SERVER_NAME:-birdnet.local}
|
||||
PHP_FPM_PORT: ${PHP_FPM_PORT:-9001}
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./www:/var/www/birdnet/
|
||||
- ./www/nginx.conf:/etc/nginx/conf.d/birdnet.conf
|
||||
context: .
|
||||
dockerfile: ./docker/nginx/Dockerfile
|
||||
args:
|
||||
- SERVER_NAME=${SERVER_NAME:-birnet.local}
|
||||
- SYMFONY_PUBLIC=/opt/birdnet/www/public
|
||||
- CHARTS_DIR=/media/birdnet/charts
|
||||
- RECORDS_DIR=/media/birdnet/records
|
||||
- PHP_FPM_HOST=birdnet_php-fpm
|
||||
- PHP_FPM_PORT=9000
|
||||
ports:
|
||||
- "81:80"
|
||||
dependends_on:
|
||||
- php
|
||||
- ${HTTP_PORT:-80}:80
|
||||
- ${HTTPS_PORT:-443}:443
|
||||
volumes:
|
||||
- birdnet_app:/opt/birdnet
|
||||
- birdnet_records:/media/data/records
|
||||
networks:
|
||||
birdnet_network:
|
||||
ipv4_address: ${IP_ADDRESS:-172.25.0.101}
|
||||
aliases:
|
||||
- ${SERVER_NAME:-birdnet.local}
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- symfony
|
||||
- php-fpm
|
||||
|
||||
birdnet:
|
||||
container_name: birdnet_analyzer
|
||||
image:
|
||||
networks:
|
||||
birdnet_network:
|
||||
driver: bridge
|
||||
ipam:
|
||||
config:
|
||||
- subnet: ${IP_SUBNET:-172.25.0.0/24}
|
||||
|
||||
volumes:
|
||||
birdnet_app:
|
||||
|
||||
birdnet_records:
|
||||
driver_opts:
|
||||
type: none
|
||||
device: ${RECORDS_DIR:-/media/data/records}
|
||||
o: bind
|
||||
|
|
|
@ -4,19 +4,24 @@ FROM debian:bullseye
|
|||
ENV REPOSITORY=${REPOSITORY:-https://github.com/UncleSamulus/BirdNET-stream.git}
|
||||
# DEBUG defaults to 1 for descriptive DEBUG logs, 0 for error logs only
|
||||
ENV DEBUG=${DEBUG:-1}
|
||||
RUN useradd birdnet
|
||||
WORKDIR /home/birdnet
|
||||
RUN useradd -m -s /bin/bash -G sudo birdnet
|
||||
USER birdnet
|
||||
|
||||
# Upgrade system
|
||||
RUN apt-get update && apt-get upgrade -y
|
||||
|
||||
# Install sudo
|
||||
# Install some dependencies
|
||||
RUN apt-get install -y \
|
||||
sudo \
|
||||
git
|
||||
git \
|
||||
curl \
|
||||
bash \
|
||||
vim \
|
||||
systemctl
|
||||
|
||||
COPY ./install.sh install.sh
|
||||
|
||||
RUN bash ./install.sh
|
||||
RUN ./install.sh
|
||||
|
||||
USER birdnet
|
||||
EXPOSE 443
|
|
@ -0,0 +1,5 @@
|
|||
CREATE DATABASE IF NOT EXISTS birdnet_default;
|
||||
CREATE DATABASE IF NOT EXISTS birdnet_observations;
|
||||
|
||||
GRANT ALL ON `birdnet_observations`.* TO 'birdnet'@'%' IDENTIFIED BY 'secret';
|
||||
GRANT ALL ON `birdnet_default`.* TO 'birdnet'@'%' IDENTIFIED BY 'secret';
|
|
@ -0,0 +1,31 @@
|
|||
FROM nginx
|
||||
|
||||
ARG SERVER_NAME
|
||||
ARG PROJECT_ROOT
|
||||
ARG SYMFONY_PUBLIC
|
||||
ARG CHARTS_DIR
|
||||
ARG RECORDS_DIR
|
||||
ARG PHP_FPM_HOST
|
||||
ARG PHP_FPM_PORT
|
||||
|
||||
RUN apt-get update && apt-get upgrade -y \
|
||||
&& apt-get install -y nginx-full
|
||||
|
||||
USER root
|
||||
COPY "docker/nginx/nginx.conf.template" "/etc/nginx/sites-available/birdnet.conf"
|
||||
RUN ln -s /etc/nginx/sites-available/birdnet.conf /etc/nginx/sites-enabled/birdnet.conf
|
||||
RUN sed -i "s|<SERVER_NAME>|${SERVER_NAME}|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<PHP_FPM_HOST>|${PHP_FPM_HOST}|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<PHP_FPM_PORT>|${PHP_FPM_PORT}|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<SYMFONY_PUBLIC>|${SYMFONY_PUBLIC}|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<RECORDS_DIR>|${RECORDS_DIR}|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<CHARTS_DIR>|${CHARTS_DIR}|g" /etc/nginx/sites-available/birdnet.conf
|
||||
RUN mkdir -p /etc/nginx/certs/birdnet
|
||||
WORKDIR /etc/nginx/certs/birdnet
|
||||
RUN openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out fullchain.pem -sha256 -days 365 -nodes --subj "/CN=${SERVER_NAME}"
|
||||
RUN sed -i "s|<CERTIFICATE>|/etc/nginx/certs/birdnet/fullchain.pem|g" /etc/nginx/sites-available/birdnet.conf \
|
||||
&& sed -i "s|<PRIVATE_KEY>|/etc/nginx/certs/birdnet/privkey.pem|g" /etc/nginx/sites-available/birdnet.conf
|
||||
|
||||
EXPOSE 443
|
||||
EXPOSE 80
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
|
@ -0,0 +1,52 @@
|
|||
server {
|
||||
listen 80;
|
||||
server_name <SERVER_NAME>;
|
||||
|
||||
location / {
|
||||
return 302 https://$host$request_uri;
|
||||
}
|
||||
|
||||
location /.well-known/acme-challenge {
|
||||
alias /var/www/html/.well-known/acme-challenge;
|
||||
allow all;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name <SERVER_NAME>;
|
||||
|
||||
fastcgi_buffers 16 16k;
|
||||
fastcgi_buffer_size 32k;
|
||||
|
||||
root <SYMFONY_PUBLIC>;
|
||||
|
||||
ssl_certificate <CERTIFICATE>;
|
||||
ssl_certificate_key <PRIVATE_KEY>;
|
||||
|
||||
index index.php;
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ /index.php$is_args$args;
|
||||
}
|
||||
|
||||
location ^~ /media/records {
|
||||
autoindex on;
|
||||
alias <RECORDS_DIR>;
|
||||
}
|
||||
|
||||
location ^~ /media/charts {
|
||||
autoindex on;
|
||||
alias <CHARTS_DIR>;
|
||||
}
|
||||
|
||||
location ~ \.php$ {
|
||||
fastcgi_split_path_info ^(.+\.php)(/.+)$;
|
||||
fastcgi_pass <PHP_FPM_HOST>:<PHP_FPM_PORT>;
|
||||
fastcgi_index index.php;
|
||||
include fastcgi.conf;
|
||||
}
|
||||
|
||||
access_log /var/log/nginx/birdnet-access.log;
|
||||
error_log /var/log/nginx/birdnet-error.log error;
|
||||
}
|
|
@ -0,0 +1,8 @@
|
|||
ARG PHP_VERSION=${PHP_VERSION:-8.1}
|
||||
|
||||
FROM php:${PHP_VERSION}-fpm
|
||||
|
||||
RUN apt-get update && apt-get upgrade -y
|
||||
RUN docker-php-ext-install pdo pdo_mysql
|
||||
|
||||
EXPOSE 9000
|
|
@ -1,16 +1,25 @@
|
|||
# Recording container for BirdNET-stream
|
||||
# Reference: https://leimao.github.io/blog/Docker-Container-Audio/
|
||||
# References:
|
||||
# - https://leimao.github.io/blog/Docker-Container-Audio/
|
||||
# - https://askubuntu.com/questions/972510/how-to-set-alsa-default-device-to-pulseaudio-sound-server-on-docker
|
||||
|
||||
FROM debian:bullseye
|
||||
|
||||
ENV DEBIAN_FRONTEND noninteractive
|
||||
|
||||
# Install packages dependencies
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install apt-utils \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
RUN apt-get update && apt-get upgrade -y \
|
||||
&& apt-get install -y \
|
||||
--no-install-recommends \
|
||||
libasound2 \
|
||||
alsa-utils \
|
||||
libsndfile1-dev \
|
||||
&& apt-get install -y ffmpeg \
|
||||
&& apt-get clean
|
||||
|
||||
RUN mkdir -p /opt/birdnet/
|
||||
WORKDIR /opt/birdnet/
|
||||
COPY config ./config
|
||||
COPY daemon/birdnet_recording.sh /usr/local/bin/birdnet_recording.sh
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/birdnet_recording.sh"]
|
|
@ -0,0 +1,75 @@
|
|||
ARG PHP_VERSION=${PHP_VERSION:-8.1}
|
||||
|
||||
FROM php:${PHP_VERSION}
|
||||
|
||||
ARG PROJECT_ROOT
|
||||
ARG NODE_VERSION
|
||||
ARG RECORDS_DIR
|
||||
ARG CHARTS_DIR
|
||||
ARG DATABASE_DEFAULT_URL
|
||||
ARG DATABASE_OBSERVATIONS_URL
|
||||
|
||||
ENV PHP_VERSION=${PHP_VERSION:-8.1} \
|
||||
NODE_VERSION=${NODE_VERSION:-16.17.0} \
|
||||
PROJECT_ROOT=${PROJECT_ROOT:-/opt/birdnet} \
|
||||
RECORDS_DIR=${RECORDS_DIR:-/media/data/birdnet/records} \
|
||||
CHARTS_DIR=${CHARTS_DIR:-/media/data/birdnet/charts} \
|
||||
DATABASE_DEFAULT_URL=${DATABASE_DEFAULT_URL:-mysql://birdnet:secret@birdnet_database/birdnet} \
|
||||
DATABASE_OBSERVATIONS_URL=${DATABASE_OBSERVATIONS_URL:-mysql://birdnet:secret@birdnet_database/birdnet_observations}
|
||||
|
||||
ENV APP_ENV=${APP_ENV:-prod}
|
||||
ENV APP_DEBUG=${APP_DEBUG:-0}
|
||||
|
||||
# RUN apt-get update && apt-get upgrade -y \
|
||||
# && apt-get install -y \
|
||||
# curl \
|
||||
# zip \
|
||||
# unzip \
|
||||
# zlib1g-dev \
|
||||
# libzip-dev \
|
||||
# git \
|
||||
# vim \
|
||||
# && apt-get clean
|
||||
|
||||
# RUN docker-php-ext-install zip pdo_mysql
|
||||
|
||||
# # Install composer
|
||||
# RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" \
|
||||
# && php composer-setup.php --install-dir=/usr/local/bin --filename=composer
|
||||
|
||||
# # Install nodejs and npm
|
||||
# ENV NVM_DIR="/usr/local/nvm"
|
||||
# RUN mkdir ${NVM_DIR}
|
||||
# RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
|
||||
# RUN . "$NVM_DIR/nvm.sh" \
|
||||
# && nvm install ${NODE_VERSION} \
|
||||
# && nvm use ${NODE_VERSION} \
|
||||
# && nvm alias default ${NODE_VERSION} \
|
||||
# && npm install -g yarn
|
||||
# ENV PATH="$PATH:/usr/local/nvm/versions/node/v${NODE_VERSION}/bin"
|
||||
|
||||
# Change permissions for the home folder of www-data (used for composer cache)
|
||||
RUN chown -R www-data:www-data /var/www
|
||||
COPY . ${PROJECT_ROOT}
|
||||
WORKDIR ${PROJECT_ROOT}/www
|
||||
RUN rm -rf {vendor,node_modules}
|
||||
RUN chown -R www-data:www-data .
|
||||
|
||||
USER www-data
|
||||
# Setup .env
|
||||
RUN cp .env.local.example .env.local
|
||||
RUN sed -i "s/^APP_ENV=.*/APP_ENV=prod/g" .env.local \
|
||||
&& sed -i "s/^APP_DEBUG=.*/APP_DEBUG=0/g" .env.local \
|
||||
&& sed -i "s/^APP_SECRET=.*/APP_SECRET=${APP_SECRET}/g" .env.local \
|
||||
&& sed -i "s|^DATABASE_DEFAULT_URL=.*|DATABASE_DEFAULT_URL=${DATABASE_DEFAULT_URL}|g" .env.local \
|
||||
&& sed -i "s|^DATABASE_OBSERVATIONS_URL=.*|DATABASE_OBSERVATIONS_URL=${DATABASE_OBSERVATIONS_URL}|g" .env.local \
|
||||
&& sed -i "s|^RECORDS_DIR=.*|RECORDS_DIR=${RECORDS_DIR}|g" .env.local \
|
||||
&& sed -i "s|^CHARTS_DIR=.*|CHARTS_DIR=${CHARTS_DIR}|g" .env.local
|
||||
|
||||
# # # Install yarn dependencies
|
||||
# RUN . "$NVM_DIR/nvm.sh" && yarn install && yarn build
|
||||
|
||||
# # Install composer dependencies
|
||||
# RUN composer install --no-interaction --prefer-dist --optimize-autoloader
|
||||
# RUN composer dump-env prod
|
||||
# RUN composer dump-autoload
|
|
@ -0,0 +1,58 @@
|
|||
# Setting up the database
|
||||
|
||||
There is two database managment systems available: sqlite or mariadb (mysql).
|
||||
|
||||
## sqlite
|
||||
|
||||
To use sqlite, simply install the sqlite3 package, if it is not already installed on the machine that runs BirdNET-stream.
|
||||
|
||||
```bash
|
||||
sudo apt-get install sqlite3
|
||||
```
|
||||
|
||||
Then fill `config/birdnet.conf` with the proper DATABASE value (you may use any database location):
|
||||
|
||||
```bash
|
||||
DATABASE="./var/db.sqlite"
|
||||
```
|
||||
|
||||
## mariadb
|
||||
|
||||
To use mariadb, you need to install the mariadb-server package.
|
||||
|
||||
```bash
|
||||
sudo apt-get install mariadb-server
|
||||
```
|
||||
|
||||
Then, populate the `config/birdnet.conf` file with the proper DATABASE uri:
|
||||
|
||||
```bash
|
||||
DATABASE="mysql://user:password@localhost/birdnet_observations"
|
||||
```
|
||||
|
||||
## Symfony configuration
|
||||
|
||||
For both method you need to adapt the file `www/.env.local` to suit your new configuration.
|
||||
|
||||
```bash
|
||||
cd www
|
||||
# If .env.local does not exists:
|
||||
cp .env.local.example .env.local
|
||||
```
|
||||
|
||||
```text
|
||||
# .env.local
|
||||
# for sqlite (example)
|
||||
DATABASE_DEFAULT_URL=sqlite:///%kernel.project_dir%/./var/db-default.sqlite
|
||||
DATABASE_OBSERVATIONS_URL=sqlite:///%kernel.project_dir%/../var/db.sqlite
|
||||
# for mariadb (example)
|
||||
DATABASE_DEFAULT_URL=mysql://user:password@localhost/birdnet_default
|
||||
DATABASE_OBSERVATIONS_URL=mysql://user:password@localhost/birdnet_observations
|
||||
```
|
||||
|
||||
## PHP modules
|
||||
|
||||
For symfony to work, make sure you have the required modules according to each method:
|
||||
|
||||
- pdo_sqlite
|
||||
- pdo_mysql
|
|
@ -0,0 +1,54 @@
|
|||
# Use docker to run BirdNET-stream
|
||||
|
||||
There are two ways to run BirdNET-stream using docker: a "all in one" container, running all services on the same container, or using a splitted approach, running each service on a separate container.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- docker
|
||||
- docker-compose (for splitted approach)
|
||||
- git
|
||||
|
||||
## Using the all in one container (not working yet)
|
||||
|
||||
The all in one container is a container that runs all services on the same container.
|
||||
|
||||
You can follow the instructions in [./docker/all/README.md](./docker/all/README.md) to create this container.
|
||||
|
||||
## Using the splitted approach (recommended)
|
||||
|
||||
The splitted approach uses docker-compose and a docker container for each service.
|
||||
|
||||
This is the recommended approach to run BirdNET-stream while using docker.
|
||||
|
||||
First of of all, you need to clone the repository.
|
||||
|
||||
```bash
|
||||
mkdir ~/Documents/BirdNET-stream
|
||||
cd ~/Documents/BirdNET-stream
|
||||
git clone -b main https://github.com/UncleSamulus/BirdNET-stream.git .
|
||||
```
|
||||
|
||||
Then, create your dotenv file and populate it with your own configuration (for instance, generate random passwords and add them to .env credentials):
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
You may need to adapt the listening ports of the services or other configuration parameters.
|
||||
In general all variables stated with ${VARIABLE:-default} inside [../docker-compose.yml](../docker-compose.yml) can be override in the .env file using `VARIABLE=value`.
|
||||
|
||||
Once that is done, you can build and start docker services:
|
||||
|
||||
```bash
|
||||
# Build images (first time only)
|
||||
docker compose build
|
||||
# Run
|
||||
docker compose up # add `-d`, to run in background
|
||||
# Stop
|
||||
docker compose down
|
||||
```
|
||||
|
||||
For a one liner:
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
14
install.sh
14
install.sh
|
@ -48,7 +48,7 @@ install_birdnetstream() {
|
|||
# Clone BirdNET-stream
|
||||
cd "$WORKDIR"
|
||||
debug "Cloning BirdNET-stream from $REPOSITORY"
|
||||
git clone -b "$BRANCH"--recurse-submodules "$REPOSITORY" .
|
||||
git clone -b "$BRANCH" --recurse-submodules "$REPOSITORY" .
|
||||
debug "Creating python3 virtual environment $PYTHON_VENV"
|
||||
python3 -m venv $PYTHON_VENV
|
||||
debug "Activating $PYTHON_VENV"
|
||||
|
@ -67,7 +67,7 @@ install_birdnetstream_services() {
|
|||
DIR="$WORKDIR"
|
||||
cd "$WORKDIR"
|
||||
debug "Setting up BirdNET stream systemd services"
|
||||
services="birdnet_recording.service birdnet_analyzis.service birdnet_miner.timer birdnet_miner.service birdnet_plotter.service birdnet_plotter.timer"
|
||||
services="birdnet_recording.service birdnet_analyzis.service birdnet_plotter.service birdnet_plotter.timer"
|
||||
read -r -a services_array <<<"$services"
|
||||
for service in ${services_array[@]}; do
|
||||
sudo cp "daemon/systemd/templates/$service" "/etc/systemd/system/"
|
||||
|
@ -78,7 +78,7 @@ install_birdnetstream_services() {
|
|||
done
|
||||
sudo sed -i "s|<VENV>|$WORKDIR/$PYTHON_VENV|g" "/etc/systemd/system/birdnet_plotter.service"
|
||||
sudo systemctl daemon-reload
|
||||
enabled_services="birdnet_recording.service birdnet_analyzis.service birdnet_miner.timer birdnet_plotter.timer"
|
||||
enabled_services="birdnet_recording.service birdnet_analyzis.service birdnet_plotter.timer"
|
||||
read -r -a services_array <<<"$services"
|
||||
for service in ${services_array[@]}; do
|
||||
debug "Enabling $service"
|
||||
|
@ -163,11 +163,11 @@ setup_http_server() {
|
|||
fi
|
||||
debug "Enable birdnet.lan domain"
|
||||
sudo ln -s /etc/nginx/sites-available/birdnet-stream.conf /etc/nginx/sites-enabled/birdnet-stream.conf
|
||||
debug "Info: Please edit /etc/nginx/sites-available/birdnet-stream.conf to set the correct server name and paths"
|
||||
debug "INFO: Please edit /etc/nginx/sites-available/birdnet-stream.conf to set the correct server name and paths"
|
||||
debug "Setup nginx variables the best way possible"
|
||||
sudo sed -i "s|<SYMFONY_PUBLIC>|$WORKDIR/www/public/|g" /etc/nginx/sites-available/birdnet-stream.conf
|
||||
sudo sed -i "s|<RECORDS_FOLDER>|$CHUNK_FOLDER/out|g" /etc/nginx/sites-available/birdnet-stream.conf
|
||||
sudo sed -i "s|<CHARTS_FOLDER>|$WORKDIR/var/charts|g" /etc/nginx/sites-available/birdnet-stream.conf
|
||||
sudo sed -i "s|<RECORDS_DIR>|$CHUNK_FOLDER/out|g" /etc/nginx/sites-available/birdnet-stream.conf
|
||||
sudo sed -i "s|<CHARTS_DIR>|$WORKDIR/var/charts|g" /etc/nginx/sites-available/birdnet-stream.conf
|
||||
debug "Generate self signed certificate"
|
||||
CERTS_LOCATION="/etc/nginx/certs/birdnet"
|
||||
sudo mkdir -p "$CERTS_LOCATION"
|
||||
|
@ -219,6 +219,8 @@ main() {
|
|||
install_web_interface
|
||||
setup_http_server
|
||||
install_config
|
||||
debug "Run loginctl enable-linger for $USER"
|
||||
loginctl enable-linger
|
||||
update_permissions
|
||||
debug "Installation done"
|
||||
}
|
||||
|
|
|
@ -27,6 +27,7 @@ uninstall_birdnet_services() {
|
|||
sudo systemctl stop "$service"
|
||||
sudo systemctl disable "$service"
|
||||
sudo rm -f "/etc/systemd/system/$service"
|
||||
sudo systemctl daemon-reload
|
||||
done
|
||||
debug "Done removing systemd services"
|
||||
}
|
||||
|
@ -36,6 +37,13 @@ uninstall_webapp() {
|
|||
debug "Removing nginx server configuration"
|
||||
sudo unlink /etc/nginx/sites-enabled/birdnet-stream.conf
|
||||
sudo systemctl restart nginx
|
||||
debug "Removing webapp directory"
|
||||
sudo rm -rf $WORKDIR
|
||||
}
|
||||
}
|
||||
|
||||
main() {
|
||||
echo "WARNING: This will remove all BirdNET-stream related files and services. \
|
||||
Note that it may forget some special configuration."
|
||||
uninstall_webapp
|
||||
uninstall_birdnet_services
|
||||
}
|
||||
|
||||
main
|
|
@ -8,7 +8,6 @@
|
|||
/public/bundles/
|
||||
/var/
|
||||
/vendor/
|
||||
yarn.lock
|
||||
###< symfony/framework-bundle ###
|
||||
|
||||
###> symfony/webpack-encore-bundle ###
|
||||
|
|
|
@ -174,26 +174,27 @@
|
|||
},
|
||||
{
|
||||
"name": "doctrine/collections",
|
||||
"version": "1.6.8",
|
||||
"version": "1.7.0",
|
||||
"source": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/doctrine/collections.git",
|
||||
"reference": "1958a744696c6bb3bb0d28db2611dc11610e78af"
|
||||
"reference": "07d15c8a766e664ec271ae84e5dfc597aeeb03b1"
|
||||
},
|
||||
"dist": {
|
||||
"type": "zip",
|
||||
"url": "https://api.github.com/repos/doctrine/collections/zipball/1958a744696c6bb3bb0d28db2611dc11610e78af",
|
||||
"reference": "1958a744696c6bb3bb0d28db2611dc11610e78af",
|
||||
"url": "https://api.github.com/repos/doctrine/collections/zipball/07d15c8a766e664ec271ae84e5dfc597aeeb03b1",
|
||||
"reference": "07d15c8a766e664ec271ae84e5dfc597aeeb03b1",
|
||||
"shasum": ""
|
||||
},
|
||||
"require": {
|
||||
"doctrine/deprecations": "^0.5.3 || ^1",
|
||||
"php": "^7.1.3 || ^8.0"
|
||||
},
|
||||
"require-dev": {
|
||||
"doctrine/coding-standard": "^9.0",
|
||||
"phpstan/phpstan": "^0.12",
|
||||
"phpstan/phpstan": "^1.4.8",
|
||||
"phpunit/phpunit": "^7.5 || ^8.5 || ^9.1.5",
|
||||
"vimeo/psalm": "^4.2.1"
|
||||
"vimeo/psalm": "^4.22"
|
||||
},
|
||||
"type": "library",
|
||||
"autoload": {
|
||||
|
@ -237,22 +238,22 @@
|
|||
],
|
||||
"support": {
|
||||
"issues": "https://github.com/doctrine/collections/issues",
|
||||
"source": "https://github.com/doctrine/collections/tree/1.6.8"
|
||||
"source": "https://github.com/doctrine/collections/tree/1.7.0"
|
||||
},
|
||||
"time": "2021-08-10T18:51:53+00:00"
|
||||
"time": "2022-08-18T05:44:45+00:00"
|
||||
},
|
||||
{
|
||||
"name": "doctrine/common",
|
||||
"version": "3.3.0",
|
||||
"version": "3.3.1",
|
||||
"source": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/doctrine/common.git",
|
||||
"reference": "c824e95d4c83b7102d8bc60595445a6f7d540f96"
|
||||
"reference": "6a76bd25b1030d35d6ba2bf2f69ca858a41fc580"
|
||||
},
|
||||
"dist": {
|
||||
"type": "zip",
|
||||
"url": "https://api.github.com/repos/doctrine/common/zipball/c824e95d4c83b7102d8bc60595445a6f7d540f96",
|
||||
"reference": "c824e95d4c83b7102d8bc60595445a6f7d540f96",
|
||||
"url": "https://api.github.com/repos/doctrine/common/zipball/6a76bd25b1030d35d6ba2bf2f69ca858a41fc580",
|
||||
"reference": "6a76bd25b1030d35d6ba2bf2f69ca858a41fc580",
|
||||
"shasum": ""
|
||||
},
|
||||
"require": {
|
||||
|
@ -261,6 +262,7 @@
|
|||
},
|
||||
"require-dev": {
|
||||
"doctrine/coding-standard": "^9.0",
|
||||
"doctrine/collections": "^1",
|
||||
"phpstan/phpstan": "^1.4.1",
|
||||
"phpstan/phpstan-phpunit": "^1",
|
||||
"phpunit/phpunit": "^7.5.20 || ^8.5 || ^9.0",
|
||||
|
@ -313,7 +315,7 @@
|
|||
],
|
||||
"support": {
|
||||
"issues": "https://github.com/doctrine/common/issues",
|
||||
"source": "https://github.com/doctrine/common/tree/3.3.0"
|
||||
"source": "https://github.com/doctrine/common/tree/3.3.1"
|
||||
},
|
||||
"funding": [
|
||||
{
|
||||
|
@ -329,20 +331,20 @@
|
|||
"type": "tidelift"
|
||||
}
|
||||
],
|
||||
"time": "2022-02-05T18:28:51+00:00"
|
||||
"time": "2022-08-20T10:48:54+00:00"
|
||||
},
|
||||
{
|
||||
"name": "doctrine/dbal",
|
||||
"version": "3.4.0",
|
||||
"version": "3.4.2",
|
||||
"source": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/doctrine/dbal.git",
|
||||
"reference": "118a360e9437e88d49024f36283c8bcbd76105f5"
|
||||
"reference": "22de295f10edbe00df74f517612f1fbd711131e2"
|
||||
},
|
||||
"dist": {
|
||||
"type": "zip",
|
||||
"url": "https://api.github.com/repos/doctrine/dbal/zipball/118a360e9437e88d49024f36283c8bcbd76105f5",
|
||||
"reference": "118a360e9437e88d49024f36283c8bcbd76105f5",
|
||||
"url": "https://api.github.com/repos/doctrine/dbal/zipball/22de295f10edbe00df74f517612f1fbd711131e2",
|
||||
"reference": "22de295f10edbe00df74f517612f1fbd711131e2",
|
||||
"shasum": ""
|
||||
},
|
||||
"require": {
|
||||
|
@ -424,7 +426,7 @@
|
|||
],
|
||||
"support": {
|
||||
"issues": "https://github.com/doctrine/dbal/issues",
|
||||
"source": "https://github.com/doctrine/dbal/tree/3.4.0"
|
||||
"source": "https://github.com/doctrine/dbal/tree/3.4.2"
|
||||
},
|
||||
"funding": [
|
||||
{
|
||||
|
@ -440,7 +442,7 @@
|
|||
"type": "tidelift"
|
||||
}
|
||||
],
|
||||
"time": "2022-08-06T20:35:57+00:00"
|
||||
"time": "2022-08-21T14:21:06+00:00"
|
||||
},
|
||||
{
|
||||
"name": "doctrine/deprecations",
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
server {
|
||||
listen 80;
|
||||
server_name birdnet.lab.home;
|
||||
server_name <SERVER_NAME>;
|
||||
|
||||
location / {
|
||||
return 302 https://$host$request_uri;
|
||||
|
@ -14,7 +14,7 @@ server {
|
|||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name birdnet.lab.home;
|
||||
server_name <SERVER_NAME>;
|
||||
|
||||
fastcgi_buffers 16 16k;
|
||||
fastcgi_buffer_size 32k;
|
||||
|
@ -32,12 +32,12 @@ server {
|
|||
|
||||
location ^~ /media/records {
|
||||
autoindex on;
|
||||
alias <RECORDS_FOLDER>;
|
||||
alias <RECORDS_DIR>;
|
||||
}
|
||||
|
||||
location ^~ /media/charts {
|
||||
autoindex on;
|
||||
alias <CHARTS_FOLDER>;
|
||||
alias <CHARTS_DIR>;
|
||||
}
|
||||
|
||||
location ~ \.php$ {
|
||||
|
|
|
@ -1,4 +1,5 @@
|
|||
<?php
|
||||
|
||||
namespace App\Controller;
|
||||
|
||||
use Symfony\Component\HttpFoundation\Response;
|
||||
|
@ -6,81 +7,124 @@ use Symfony\Component\HttpFoundation\Request;
|
|||
use Symfony\Component\Routing\Annotation\Route;
|
||||
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
|
||||
use App\AppBundle\ConnectionObservations;
|
||||
use Psr\Log\LoggerInterface;
|
||||
use Symfony\Component\HttpKernel\Log\Logger;
|
||||
|
||||
class HomeController extends AbstractController
|
||||
{
|
||||
private ConnectionObservations $connection;
|
||||
private LoggerInterface $logger;
|
||||
|
||||
public function __construct(ConnectionObservations $connection)
|
||||
public function __construct(ConnectionObservations $connection, LoggerInterface $logger)
|
||||
{
|
||||
$this->connection = $connection;
|
||||
$this->logger = $logger;
|
||||
}
|
||||
|
||||
/**
|
||||
* @Route("", name="home")
|
||||
* @Route("/{_locale<%app.supported_locales%>}/", name="home_i18n")
|
||||
*/
|
||||
public function index()
|
||||
public function index(Request $request)
|
||||
{
|
||||
$date = $request->get("on");
|
||||
if ($date == null) {
|
||||
$date = date("Y-m-d");
|
||||
}
|
||||
return $this->render('index.html.twig', [
|
||||
"stats" => $this->get_stats(),
|
||||
"charts" => $this->last_chart_generated(),
|
||||
"stats" => $this->get_stats($date),
|
||||
"charts" => $this->last_chart_generated($date),
|
||||
]);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @Route("/about", name="about")
|
||||
* @Route("/{_locale<%app.supported_locales%>}/about", name="about_i18n")
|
||||
*/
|
||||
public function about()
|
||||
{
|
||||
return $this->render('about/index.html.twig', [
|
||||
|
||||
]);
|
||||
return $this->render('about/index.html.twig', []);
|
||||
}
|
||||
|
||||
private function get_stats()
|
||||
private function get_stats($date)
|
||||
{
|
||||
$stats = array();
|
||||
$stats["most-recorded-species"] = $this->get_most_recorded_species();
|
||||
$stats["last-detected-species"] = $this->get_last_recorded_species();
|
||||
$stats["number-of-species-detected"] = $this->get_number_of_species_detected($date);
|
||||
return $stats;
|
||||
}
|
||||
|
||||
private function get_most_recorded_species()
|
||||
private function get_most_recorded_species()
|
||||
{
|
||||
$species = [];
|
||||
$sql = "SELECT `scientific_name`, `common_name`, COUNT(*) AS contact_count
|
||||
FROM `taxon`
|
||||
INNER JOIN `observation`
|
||||
ON `taxon`.`taxon_id` = `observation`.`taxon_id`
|
||||
ORDER BY `contact_count` DESC LIMIT 1";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$result = $stmt->executeQuery();
|
||||
$species = $result->fetchAllAssociative();
|
||||
return $species[0];
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$result = $stmt->executeQuery();
|
||||
$species = $result->fetchAllAssociative()[0];
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return $species;
|
||||
}
|
||||
|
||||
private function get_last_recorded_species()
|
||||
private function get_last_recorded_species()
|
||||
{
|
||||
$species = [];
|
||||
$sql = "SELECT `scientific_name`, `common_name`, `date`, `audio_file`, `confidence`
|
||||
FROM `observation`
|
||||
INNER JOIN `taxon`
|
||||
ON `observation`.`taxon_id` = `taxon`.`taxon_id`
|
||||
ORDER BY `date` DESC LIMIT 1";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$result = $stmt->executeQuery();
|
||||
$species = $result->fetchAllAssociative();
|
||||
return $species[0];
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$result = $stmt->executeQuery();
|
||||
$species = $result->fetchAllAssociative()[0];
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return $species;
|
||||
}
|
||||
|
||||
private function last_chart_generated() {
|
||||
|
||||
private function get_number_of_species_detected($date)
|
||||
{
|
||||
$count = 0;
|
||||
$sql = "SELECT COUNT(`taxon_id`) AS contact_count
|
||||
FROM `observation`
|
||||
WHERE STRFTIME('%Y-%m-%d', `date`) = :date
|
||||
GROUP BY `taxon_id`";
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(":date", $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$output = $result->fetchAllAssociative();
|
||||
if ($output != null) {
|
||||
$count = $output[0]["contact_count"];
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return $count;
|
||||
}
|
||||
|
||||
private function last_chart_generated()
|
||||
{
|
||||
$files = glob($this->getParameter('kernel.project_dir') . '/../var/charts/*.png');
|
||||
usort($files, function($a, $b) {
|
||||
return filemtime($b) - filemtime($a);
|
||||
});
|
||||
$last_chart = basename(array_pop($files));
|
||||
return $last_chart;
|
||||
if (count($files) > 0) {
|
||||
usort($files, function ($a, $b) {
|
||||
return filemtime($a) - filemtime($b);
|
||||
});
|
||||
|
||||
$last_chart = basename(array_pop($files));
|
||||
return $last_chart;
|
||||
} else {
|
||||
$this->logger->info("No charts found");
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
|
|
@ -1,39 +0,0 @@
|
|||
<?php
|
||||
|
||||
namespace App\Controller;
|
||||
|
||||
use Symfony\Component\HttpFoundation\Response;
|
||||
use Symfony\Component\HttpFoundation\Request;
|
||||
use Symfony\Component\Routing\Annotation\Route;
|
||||
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
|
||||
|
||||
class LogsController extends AbstractController
|
||||
{
|
||||
private $allowed_services = "recording analyzis miner plotter";
|
||||
/**
|
||||
* @Route("/logs/{service}", name="logs")
|
||||
* @Route("/{_locale<%app.supported_locales%>}/logs/{service}", name="logs_i18n")
|
||||
*/
|
||||
public function logs($service = "all")
|
||||
{
|
||||
$logs = "";
|
||||
if ($service === "all") {
|
||||
foreach (explode(" ", $this->allowed_services) as $service) {
|
||||
$logs .= $this->journal_logs($service);
|
||||
}
|
||||
} else if (str_contains($this->allowed_services, $service)) {
|
||||
$logs .= $this->journal_logs($service);
|
||||
} else {
|
||||
return new Response("Service not found", Response::HTTP_BAD_REQUEST);
|
||||
}
|
||||
return $this->render('logs/logs.html.twig', [
|
||||
'logs' => $logs
|
||||
]);
|
||||
}
|
||||
|
||||
private function journal_logs($service)
|
||||
{
|
||||
$logs = shell_exec("journalctl -u birdnet_recording -n 10");
|
||||
return $logs;
|
||||
}
|
||||
}
|
|
@ -6,15 +6,20 @@ use Symfony\Component\HttpFoundation\Request;
|
|||
use Symfony\Component\Routing\Annotation\Route;
|
||||
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
|
||||
use App\AppBundle\ConnectionObservations;
|
||||
use Psr\Log\LoggerInterface;
|
||||
|
||||
class TodayController extends AbstractController
|
||||
{ private ConnectionObservations $connection;
|
||||
{
|
||||
private ConnectionObservations $connection;
|
||||
private LoggerInterface $logger;
|
||||
|
||||
public function __construct(ConnectionObservations $connection)
|
||||
public function __construct(ConnectionObservations $connection, LoggerInterface $logger)
|
||||
{
|
||||
$this->connection = $connection;
|
||||
$this->logger = $logger;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @Route("/today", name="today")
|
||||
* @Route("/{_locale<%app.supported_locales%>}/today", name="today_i18n")
|
||||
|
@ -88,28 +93,38 @@ class TodayController extends AbstractController
|
|||
|
||||
private function recorded_species_by_date($date)
|
||||
{
|
||||
$recorded_species = [];
|
||||
$sql = "SELECT `taxon`.`taxon_id`, `scientific_name`, `common_name`, COUNT(*) AS `contact_count`, MAX(`confidence`) AS max_confidence
|
||||
FROM observation
|
||||
INNER JOIN taxon
|
||||
ON observation.taxon_id = taxon.taxon_id
|
||||
WHERE strftime('%Y-%m-%d', `observation`.`date`) = :date
|
||||
GROUP BY observation.taxon_id";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
return $result->fetchAllAssociative();
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$recorded_species = $result->fetchAllAssociative();
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return $recorded_species;
|
||||
}
|
||||
|
||||
private function recorded_species_by_id_and_date($id, $date)
|
||||
{
|
||||
/* Get taxon even if there is no record this date */
|
||||
$sql = "SELECT * FROM `taxon` WHERE `taxon_id` = :id";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$result = $stmt->executeQuery();
|
||||
$taxon = $result->fetchAllAssociative()[0];
|
||||
if (!$taxon) {
|
||||
return [];
|
||||
$taxon = [];
|
||||
$stat = [];
|
||||
$records = [];
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$result = $stmt->executeQuery();
|
||||
$taxon = $result->fetchAllAssociative()[0];
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
/* Get daily stats */
|
||||
$sql = "SELECT COUNT(*) AS `contact_count`, MAX(`confidence`) AS `max_confidence`
|
||||
|
@ -118,33 +133,47 @@ class TodayController extends AbstractController
|
|||
ON `taxon`.`taxon_id` = `observation`.`taxon_id`
|
||||
WHERE strftime('%Y-%m-%d', `observation`.`date`) = :date
|
||||
AND `observation`.`taxon_id` = :id";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$stat = $result->fetchAllAssociative();
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$stat = $result->fetchAllAssociative();
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
$sql = "SELECT * FROM `observation`
|
||||
WHERE `taxon_id` = :id
|
||||
AND strftime('%Y-%m-%d', `observation`.`date`) = :date
|
||||
ORDER BY `observation`.`date` ASC";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$records = $result->fetchAllAssociative();
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$records = $result->fetchAllAssociative();
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return array("taxon" => $taxon, "stat" => $stat, "records" => $records);
|
||||
}
|
||||
|
||||
private function best_confidence_today($id, $date)
|
||||
{
|
||||
$best_confidence = 0;
|
||||
$sql = "SELECT MAX(`confidence`) AS confidence
|
||||
FROM `observation`
|
||||
WHERE strftime('%Y-%m-%d', `observation`.`date`) = :date
|
||||
AND `taxon_id` = :id";
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
return $result->fetchAllAssociative();
|
||||
try {
|
||||
$stmt = $this->connection->prepare($sql);
|
||||
$stmt->bindValue(':id', $id);
|
||||
$stmt->bindValue(':date', $date);
|
||||
$result = $stmt->executeQuery();
|
||||
$result->fetchAllAssociative();
|
||||
} catch (\Exception $e) {
|
||||
$this->logger->error($e->getMessage());
|
||||
}
|
||||
return $best_confidence;
|
||||
}
|
||||
}
|
|
@ -41,10 +41,9 @@
|
|||
<li class="dropdown">
|
||||
<span class="dropdown-toggle">{{ "Tools"|trans }}</span>
|
||||
<ul class="dropdown-content">
|
||||
{% include 'utils/nav-item.html.twig' with {
|
||||
route: 'logs',
|
||||
text: 'View Logs'|trans
|
||||
} %}
|
||||
<li><a href="/ttyd">
|
||||
{{ "Logs"|trans }}
|
||||
</a></li>
|
||||
{% include 'utils/nav-item.html.twig' with {
|
||||
route: 'services_status',
|
||||
text: 'Status'|trans
|
||||
|
|
|
@ -1,50 +1,78 @@
|
|||
<div id="stats">
|
||||
<h2>{{ "Quick Stats" | trans }}</h2>
|
||||
<ul>
|
||||
<li class="most-recorded-species">
|
||||
{{ "Most recorded species" | trans }}:
|
||||
{% if stats["most-recorded-species"] is defined %}
|
||||
<span class="scientific-name">
|
||||
{{ stats["most-recorded-species"]["scientific_name"] }}
|
||||
</span>
|
||||
(<span class="common_name">{{ stats["most-recorded-species"]["common_name"] }}</span>)
|
||||
{{ "with" | trans }}
|
||||
<span class="observation-count">
|
||||
{{ stats["most-recorded-species"]["contact_count"] }}
|
||||
</span>
|
||||
{{ "contacts" | trans }}.
|
||||
{% else %}
|
||||
{{ "No species in database." | trans }}
|
||||
{% endif %}
|
||||
</li>
|
||||
<li class="last-recorded-species">
|
||||
{{ "Last detected species" | trans }}:
|
||||
{% if stats["last-detected-species"] is defined %}
|
||||
<span class="scientific-name">
|
||||
{{ stats["last-detected-species"]["scientific_name"] }}
|
||||
</span>
|
||||
(<span class="common_name">{{ stats["last-detected-species"]["common_name"] }}</span>)
|
||||
{{ "with" | trans }}
|
||||
<span class="confidence">
|
||||
{{ stats["last-detected-species"]["confidence"] }}
|
||||
</span>
|
||||
{{ "AI confidence" | trans }}
|
||||
<span class="datetime">
|
||||
{% set date = stats["last-detected-species"]["date"] %}
|
||||
{% if date | date("Y-m-d") == "now" | date("Y-m-d") %}
|
||||
{{ "today" | trans }}
|
||||
{% else %}
|
||||
{{ "on" | trans }}
|
||||
{{ date | format_datetime("full", "none") }}
|
||||
{% endif %}
|
||||
at
|
||||
<span class="time">
|
||||
{{ date | date("H:i") }}
|
||||
</span>
|
||||
</span>.
|
||||
{% else %}
|
||||
{{ "No species in database" | trans }}
|
||||
{% endif %}
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<h2>
|
||||
{{ 'Quick Stats'|trans }}
|
||||
</h2>
|
||||
<ul>
|
||||
<li class="stat">
|
||||
{{ 'Most recorded species'|trans }}:{% if
|
||||
stats['most-recorded-species'] is defined
|
||||
and (stats['most-recorded-species']|length) > 0 %}
|
||||
<span class="scientific-name">
|
||||
{{ stats['most-recorded-species']['scientific_name'] }}
|
||||
</span>
|
||||
(<span class="common_name">
|
||||
{{ stats['most-recorded-species']['common_name'] }}
|
||||
</span>)
|
||||
{{ 'with'|trans }}
|
||||
<span class="observation-count">
|
||||
{{ stats['most-recorded-species']['contact_count'] }}
|
||||
</span>
|
||||
{{ 'contacts'|trans }}.
|
||||
{% else %}
|
||||
{{ 'No species in database.'|trans }}
|
||||
{% endif %}
|
||||
</li>
|
||||
<li class="stat">
|
||||
{{ 'Last detected species'|trans }}:{% if
|
||||
stats['last-detected-species'] is defined
|
||||
and (stats['last-detected-species']|length) > 0 %}
|
||||
<span class="scientific-name">
|
||||
{{ stats['last-detected-species']['scientific_name'] }}
|
||||
</span>
|
||||
(<span class="common_name">
|
||||
{{ stats['last-detected-species']['common_name'] }}
|
||||
</span>)
|
||||
{{ 'with'|trans }}
|
||||
<span class="confidence">
|
||||
{{ stats['last-detected-species']['confidence'] }}
|
||||
</span>
|
||||
{{ 'AI confidence'|trans }}
|
||||
<span class="datetime">
|
||||
{% set date = stats['last-detected-species']['date'] %}
|
||||
{% if (date|date('Y-m-d')) == ('now'|date('Y-m-d')) %}
|
||||
{{ 'today'|trans }}
|
||||
{% else %}
|
||||
{{ 'on'|trans }}
|
||||
{{ date|format_datetime('full', 'none') }}
|
||||
{% endif %}at
|
||||
<span class="time">{{ date|date('H:i') }}</span>
|
||||
</span>.
|
||||
{% else %}
|
||||
{{ 'No species in database'|trans }}
|
||||
{% endif %}
|
||||
</li>
|
||||
<li class="stat">
|
||||
{% set today = 'now'|date('Y-m-d') %}
|
||||
{% set date = app.request.get('on') %}
|
||||
{% if
|
||||
stats['number-of-species-detected'] is defined
|
||||
and stats['number-of-species-detected'] > 0 %}
|
||||
{% if today == date %}
|
||||
{{ 'Number of species detected today: '|trans }}
|
||||
{% else %}
|
||||
{{ 'Number of species detected on '|trans }}
|
||||
{{ date|format_datetime('full', 'none') }}:
|
||||
{% endif %}
|
||||
<span>{{ stats['number-of-species-detected'] }}</span>.
|
||||
{% else %}
|
||||
{# {{ 'No species detected today'|trans }} #}
|
||||
{% if today == date %}
|
||||
{{ 'No species detected today.'|trans }}
|
||||
{% else %}
|
||||
{{ 'No species detected on '|trans }}
|
||||
{{ date|format_datetime('full', 'none') }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
|
Loading…
Reference in New Issue