Compare commits

..

3 Commits

Author SHA1 Message Date
Matthias
7d297ed805
fixup! correct typos, adapt chmod of bash scripts
Signed-off-by: Matthias <matthias@pebble>
2024-10-24 01:10:06 +02:00
Matthias
c8404cdd54
add panormax cli
Signed-off-by: Matthias <matthias@pebble>
2024-10-24 00:01:05 +02:00
Matthias
5099f0f368
correct typos, adapt chmod of bash scripts
Signed-off-by: Matthias <matthias@pebble>
2024-10-22 23:12:54 +02:00
5 changed files with 24 additions and 10 deletions

View File

@ -3,9 +3,9 @@ Simple code to download images in one or several mapillary sequences. The images
## How to use ## How to use
### Setup env variables ### Setup env variables
copy the secret variables files to setup your maipllary dev token, and your panoramax OpenStreetMap token. copy the secret variables files to setup your mapillary dev token, and your panoramax OpenStreetMap token.
```Shell ```Shell
cp secrets_variables_examples.sh secrets_variables.sh cp secrets_variables_example.sh secrets_variables.sh
editor secrets_variables.sh editor secrets_variables.sh
``` ```
@ -13,10 +13,13 @@ editor secrets_variables.sh
To avoid to download everything at once, this is a 2 steps process: To avoid to download everything at once, this is a 2 steps process:
- 1/ We set a list of users for which we want to get the list of sequences to download, and get a bash script for each user. - 1/ We set a list of users for which we want to get the list of sequences to download, and get a bash script for each user.
Change the list of users in the batch file `batch_get_username.sh`. Change the list of users in the batch file `batch_get_username.sh`.
change the access token with your access token and the sequence ids with the ids of the sequences you want to download
```bash ```bash
usernames=( "someone_having_nice_pictures" "someone_else" "oh_look_a_these_usernames" ) # use real accounts, it might work better usernames=( "someone_having_nice_pictures" "someone_else" "oh_look_a_these_usernames" ) # use real accounts, it might work better
``` ```
then, execute:
```bash
./batch_get_username.sh
```
This will give you a few files that the other scripts will use to generate a script to download all sequences of each user. One user, one bash script generated. This will give you a few files that the other scripts will use to generate a script to download all sequences of each user. One user, one bash script generated.
@ -25,7 +28,7 @@ To avoid to download everything at once, this is a 2 steps process:
bash script_bash_get_sequences_for_user_SOMEONE.sh bash script_bash_get_sequences_for_user_SOMEONE.sh
``` ```
**Or all of the generated batch scripts at once.** **Or all of the generated batch scripts at once.**
```bash ```bash
bash script_bash_get_sequences_for_user_* bash script_bash_get_sequences_for_user_*
``` ```
@ -99,9 +102,10 @@ git clone https://github.com/Stefal/mapillary_download.git
cd mapillary_download cd mapillary_download
python3 -m venv mly_venv python3 -m venv mly_venv
source mly_venv/bin/activate source mly_venv/bin/activate
source secrets_variables.sh
python -m pip install -r requirements.txt python -m pip install -r requirements.txt
``` ```
Then you can run `python mapillary_download "MLY|xxxx|xxxxxxx" --sequence_ids xxxxxxxxxxx` Then you can run `python mapillary_download.py $MAPILLARY_DEV_TOKEN --sequence_ids xxxxxxxxxxx`
When you're done with the script, simply run `deactivate` to exit the virtual environnement. When you're done with the script, simply run `deactivate` to exit the virtual environnement.
On windows you can create a virtual environnement too, or use the prebuilt `mapillary_download.exe` available on the release page. On windows you can create a virtual environnement too, or use the prebuilt `mapillary_download.exe` available on the release page.
@ -114,8 +118,10 @@ go in your `data` folder, and to the user subfolder, and assuming you have alrea
```Shell ```Shell
source mly_venv/bin/activate
source secrets_variables.sh
cd data/some_user cd data/some_user
for dir in */; do dir=${dir%?} ; geovisio upload --api-url https://panoramax.openstreetmap.fr "$dir" --token=BLAH_BLAH ; done for dir in */; do dir=${dir%?} ; panoramax_cli upload --api-url https://panoramax.openstreetmap.fr "$dir" --token=$PANORAMAX_DEV_TOKEN ; done
``` ```
Have fun! Have fun!

View File

@ -7,9 +7,9 @@ import argparse
def parse_args(argv =None): def parse_args(argv =None):
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
parser.add_argument('--username', type=str, help='Username to get the sequences id of') parser.add_argument('--username', type=str, help='Username to get the sequences id of', required=True)
parser.add_argument('--dev_token', type=str, help='Your mapillary developer token') parser.add_argument('--dev_token', type=str, help='Your mapillary developer token')
parser.add_argument('--max_sequence', type=str, help='Username to get the sequences id of') parser.add_argument('--max_sequence', type=str, help='Limit the amount of retrieved sequence ids')
global args global args
args = parser.parse_args(argv) args = parser.parse_args(argv)

1
get_user.sh Normal file → Executable file
View File

@ -9,6 +9,7 @@ export username=$1
export num_user=$2 export num_user=$2
echo "télécharger la séquence pour l'utilisateur $username, $num_user" echo "télécharger la séquence pour l'utilisateur $username, $num_user"
bash curl_land.sh "$username" "$num_user" > "out_${username}.json" bash curl_land.sh "$username" "$num_user" > "out_${username}.json"
#FIXME curl_land script not included in this repo
echo "séquences récupérées:" echo "séquences récupérées:"
num_sequences=$(grep -o -w 'image_id' "out_${username}.json" | wc -l) num_sequences=$(grep -o -w 'image_id' "out_${username}.json" | wc -l)

View File

@ -1,4 +1,5 @@
requests >= 2.28.2 requests >= 2.28.2
pytz >= 2023.3 pytz >= 2023.3
timezonefinder >=6.2.0 timezonefinder >=6.2.0
pyexiv2 >= 2.8.2 pyexiv2 >= 2.8.2
panoramax_cli >= 1.1.1

View File

@ -1,5 +1,11 @@
# scripts common shared variables # scripts common shared variables
# this is to copy as a file named "secrets_variables.sh" which is ignored in this git repo # this is to copy as a file named "secrets_variables.sh" which is ignored in this git repo
# Mapillary:
# register an application at https://www.mapillary.com/dashboard/developers
# read permissions needed, all other fields can be random. copy the client token
export MAPILLARY_DEV_TOKEN="MLY|blahblah_replace_it" export MAPILLARY_DEV_TOKEN="MLY|blahblah_replace_it"
# Panoramax
# copy the token from your profile settings at https://panoramax.openstreetmap.fr/settings
export PANORAMAX_DEV_TOKEN="yourtokenhere"