Commit 76dcdfe1 authored by Andrey S. Petrov's avatar Andrey S. Petrov

Initial commit

parents
File added
1.png

326 KB

2.png

239 KB

3.png

431 KB

File added
## Общее описание решения
Проект является цифровым решением в рамках конкурса World AI&DATA Challenge, соответствующиим задаче "Forecasting water levels during the spring flood" (https://git.asi.ru/tasks/world-ai-and-data-challenge/forecasting-water-levels-during-the-spring-flood)
## Общее описание логики работы решения
С помощью скриптов расположенных в папке neuro генеририруется готовая модель машинного обучения. По результатам ее работы строится карта текущих прогнозов и передается на для показа в веб-версии, находящейся в папке front и реализованной с помощтю React.js.
![Screenshot](1.png)
![Screenshot](2.png)
## Требования к окружению для запуска продукта
Платформа: кроссплатформенное решение, linux, windows, macOS, иное.
Используемый язык программирования python 3.6+.
## Сценарий сборки и запуска проекта
Для запуска веб версии необходимо установить все пакеты командой yarn или npm i. И после запустить проект командой yarn start.
Для работы бек версии находящейся в папке neuro необходимо установить следующие зависимости командой pip install:
- absl-py==0.9.0
- aenum==2.2.3
- affine==2.3.0
- appdirs==1.4.3
- astor==0.7.1
- astunparse==1.6.3
- attrs==19.3.0
- blinker==1.4
- bolt-python==0.7.1
- boto==2.49.0
- boto3==1.12.37
- botocore==1.15.37
- cachetools==3.1.1
- Cerberus==1.3.2
- certifi==2020.4.5.1
- cffi==1.14.0
- chardet==3.0.4
- click==7.1.1
- click-plugins==1.1.1
- cligj==0.5.0
- cryptography==2.8
- cycler==0.10.0
- decorator==4.4.2
- descartes==1.1.0
- docutils==0.15.2
- earthpy==0.9.1
- eo-learn==0.7.3
- eo-learn-core==0.7.3
- eo-learn-coregistration==0.7.3
- eo-learn-features==0.7.3
- eo-learn-geometry==0.7.3
- eo-learn-io==0.7.3
- eo-learn-mask==0.7.3
- eo-learn-ml-tools==0.7.3
- eo-learn-visualization==0.7.3
- Fiona==1.8.13
- fs==2.4.11
- fs-s3fs==1.1.1
- future==0.18.2
- gast==0.2.2
- GDAL==3.0.4
- gdal2tiles==0.1.7
- geojson==2.5.0
- geopandas==0.7.0
- google-auth==1.12.0
- google-auth-oauthlib==0.4.1
- google-pasta==0.2.0
- graphviz==0.13.2
- grpcio==1.27.2
- h5py==2.10.0
- homura==0.1.5
- humanize==2.2.0
- idna==2.9
- imagecodecs==2020.2.18
- imageio==2.8.0
- Jinja2==2.11.1
- jmespath==0.9.5
- joblib==0.14.1
- jsonpickle==1.3
- Keras==2.3.1
- Keras-Applications==1.0.8
- keras-contrib==2.0.8
- Keras-Preprocessing==1.1.2
- kiwisolver==1.1.0
- label-maker==0.7.0
- lightgbm==2.3.1
- llvmlite==0.31.0
- Mako==1.1.0
- mapbox-vector-tile==1.2.0
- Markdown==3.2.1
- MarkupSafe==1.1.1
- matplotlib==3.2.1
- mbutil==0.3.0
- mercantile==1.1.2
- mkl-service==2.3.0
- munch==2.5.0
- networkx==2.4
- numba==0.48.0
- numpy==1.18.1
- oauthlib==3.0.1
- olefile==0.46
- opencv-contrib-python-headless==4.2.0.34
- opt-einsum==3.2.1
- pandas==1.0.3
- patsy==0.5.1
- Pillow==7.0.0
- protobuf==3.11.4
- pyasn1==0.4.8
- pyasn1-modules==0.2.7
- pyclipper==1.1.0.post1
- pycparser==2.20
- pycurl==7.43.0.5
- pydot==1.4.1
- Pygments==2.6.1
- pygpu==0.7.6
- PyJWT==1.7.1
- pyOpenSSL==19.1.0
- pyparsing==2.4.6
- pyproj==2.6.0
- PyQt5==5.12.3
- PyQt5-sip==4.19.18
- PyQtWebEngine==5.12.1
- pyreadline==2.1
- pyshp==2.1.0
- PySocks==1.7.1
- python-dateutil==2.8.1
- pytz==2019.3
- PyWavelets==1.1.1
- PyYAML==5.3.1
- rasterio==1.1.3
- requests==2.23.0
- requests-oauthlib==1.2.0
- rsa==4.0
- Rtree==0.9.4
- s2cloudless==1.4.0
- s3transfer==0.3.3
- scikit-image==0.16.2
- scikit-learn==0.22.2.post1
- scipy==1.4.1
- seaborn==0.10.0
- sentinelhub==3.0.2
- Shapely==1.7.0
- six==1.14.0
- snuggs==1.4.7
- statsmodels==0.11.1
- tb-nightly==2.3.0a20200518
- tensorboard==2.2.1
- tensorboard-plugin-wit==1.6.0.post3
- tensorflow==2.1.0
- tensorflow-estimator==2.1.0
- termcolor==1.1.0
- tf-estimator-nightly==2.3.0.dev2020051801
- tf-nightly==2.3.0.dev20200518
- Theano==1.0.4
- thunder-python==1.4.2
- thunder-registration==1.0.1
- tifffile==2020.2.16
- tilepie==0.2.1
- tornado==6.0.4
- tqdm==4.45.0
- urllib3==1.25.7
- utm==0.5.0
- Werkzeug==0.16.1
- win-inet-pton==1.1.0
- wincertstore==0.2
- wrapt==1.12.1
После этого запустить скрипт командой python3 *script_name*
## Примеры использования
![Screenshot](3.png)
## Используемые наборы данных
ASTER Global Digital Elevation Map Announcement (https://asterweb.jpl.nasa.gov/gdem.asp)
NSIDC (https://nsidc.org/)
Copernicus Open Access Hub (https://scihub.copernicus.eu/)
## Дополнительный инструментарий
Предварительные данные для расчетов подготавливаются по средством инструмента SNAP 7.0 (https://step.esa.int/main/download/snap-download/)
## General Solution Description
The project is the digital solution for a World AI&DATA Challenge's task "Forecasting water levels during the spring flood" (https://git.asi.ru/tasks/world-ai-and-data-challenge/forecasting-water-levels-during-the-spring-flood)
## Solution's logics general description
With the help of scripts, a ready-made machine learning model is generated in the neuron. Based on the results of its work, a map of current predictions and results is built for the web version, which is located in the folder and is implemented using React.js.
![Screenshot](1.png)
![Screenshot](2.png)
## Execution environmental requirements and setup
Platform: cross-platform solution, linux, windows, macOS, others.
The programming language used is python 3.6+.
## Compilation and launch scenario
To run the web version, you need to install all packages with the yarn command or npm i. And then start the project with the command yarn start.
For the back version of the neuro folder to work, you need to install the following dependencies with the pip install command:
- absl-py==0.9.0
- aenum==2.2.3
- affine==2.3.0
- appdirs==1.4.3
- astor==0.7.1
- astunparse==1.6.3
- attrs==19.3.0
- blinker==1.4
- bolt-python==0.7.1
- boto==2.49.0
- boto3==1.12.37
- botocore==1.15.37
- cachetools==3.1.1
- Cerberus==1.3.2
- certifi==2020.4.5.1
- cffi==1.14.0
- chardet==3.0.4
- click==7.1.1
- click-plugins==1.1.1
- cligj==0.5.0
- cryptography==2.8
- cycler==0.10.0
- decorator==4.4.2
- descartes==1.1.0
- docutils==0.15.2
- earthpy==0.9.1
- eo-learn==0.7.3
- eo-learn-core==0.7.3
- eo-learn-coregistration==0.7.3
- eo-learn-features==0.7.3
- eo-learn-geometry==0.7.3
- eo-learn-io==0.7.3
- eo-learn-mask==0.7.3
- eo-learn-ml-tools==0.7.3
- eo-learn-visualization==0.7.3
- Fiona==1.8.13
- fs==2.4.11
- fs-s3fs==1.1.1
- future==0.18.2
- gast==0.2.2
- GDAL==3.0.4
- gdal2tiles==0.1.7
- geojson==2.5.0
- geopandas==0.7.0
- google-auth==1.12.0
- google-auth-oauthlib==0.4.1
- google-pasta==0.2.0
- graphviz==0.13.2
- grpcio==1.27.2
- h5py==2.10.0
- homura==0.1.5
- humanize==2.2.0
- idna==2.9
- imagecodecs==2020.2.18
- imageio==2.8.0
- Jinja2==2.11.1
- jmespath==0.9.5
- joblib==0.14.1
- jsonpickle==1.3
- Keras==2.3.1
- Keras-Applications==1.0.8
- keras-contrib==2.0.8
- Keras-Preprocessing==1.1.2
- kiwisolver==1.1.0
- label-maker==0.7.0
- lightgbm==2.3.1
- llvmlite==0.31.0
- Mako==1.1.0
- mapbox-vector-tile==1.2.0
- Markdown==3.2.1
- MarkupSafe==1.1.1
- matplotlib==3.2.1
- mbutil==0.3.0
- mercantile==1.1.2
- mkl-service==2.3.0
- munch==2.5.0
- networkx==2.4
- numba==0.48.0
- numpy==1.18.1
- oauthlib==3.0.1
- olefile==0.46
- opencv-contrib-python-headless==4.2.0.34
- opt-einsum==3.2.1
- pandas==1.0.3
- patsy==0.5.1
- Pillow==7.0.0
- protobuf==3.11.4
- pyasn1==0.4.8
- pyasn1-modules==0.2.7
- pyclipper==1.1.0.post1
- pycparser==2.20
- pycurl==7.43.0.5
- pydot==1.4.1
- Pygments==2.6.1
- pygpu==0.7.6
- PyJWT==1.7.1
- pyOpenSSL==19.1.0
- pyparsing==2.4.6
- pyproj==2.6.0
- PyQt5==5.12.3
- PyQt5-sip==4.19.18
- PyQtWebEngine==5.12.1
- pyreadline==2.1
- pyshp==2.1.0
- PySocks==1.7.1
- python-dateutil==2.8.1
- pytz==2019.3
- PyWavelets==1.1.1
- PyYAML==5.3.1
- rasterio==1.1.3
- requests==2.23.0
- requests-oauthlib==1.2.0
- rsa==4.0
- Rtree==0.9.4
- s2cloudless==1.4.0
- s3transfer==0.3.3
- scikit-image==0.16.2
- scikit-learn==0.22.2.post1
- scipy==1.4.1
- seaborn==0.10.0
- sentinelhub==3.0.2
- Shapely==1.7.0
- six==1.14.0
- snuggs==1.4.7
- statsmodels==0.11.1
- tb-nightly==2.3.0a20200518
- tensorboard==2.2.1
- tensorboard-plugin-wit==1.6.0.post3
- tensorflow==2.1.0
- tensorflow-estimator==2.1.0
- termcolor==1.1.0
- tf-estimator-nightly==2.3.0.dev2020051801
- tf-nightly==2.3.0.dev20200518
- Theano==1.0.4
- thunder-python==1.4.2
- thunder-registration==1.0.1
- tifffile==2020.2.16
- tilepie==0.2.1
- tornado==6.0.4
- tqdm==4.45.0
- urllib3==1.25.7
- utm==0.5.0
- Werkzeug==0.16.1
- win-inet-pton==1.1.0
- wincertstore==0.2
- wrapt==1.12.1
After that, run the script with the command python3 * script_name *
## Use cases and examples
![Screenshot](3.png)
## Used datasets
ASTER Global Digital Elevation Map Announcement (https://asterweb.jpl.nasa.gov/gdem.asp)
NSIDC (https://nsidc.org/)
Copernicus Open Access Hub (https://scihub.copernicus.eu/)
## Additional instrumentation and tools
Preliminary data for calculations are prepared using the SNAP 7.0 tool (https://step.esa.int/main/download/snap-download/)
\ No newline at end of file
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# production
/build
# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
{
"name": "geohack",
"version": "0.1.0",
"private": true,
"dependencies": {
"@material-ui/core": "^4.11.0",
"@material-ui/icons": "^4.9.1",
"@testing-library/jest-dom": "^4.2.4",
"@testing-library/react": "^9.3.2",
"@testing-library/user-event": "^7.1.2",
"leaflet": "^1.6.0",
"leaflet-tilelayer-colorizr": "^1.0.7",
"react": "^16.13.1",
"react-dom": "^16.13.1",
"react-leaflet": "^2.7.0",
"react-scripts": "3.4.1",
"styled-components": "^5.1.1"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": "react-app"
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
}
}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/leaflet/1.3.1/leaflet.css">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style>
.leaflet-container {
height: 90.5vh;
width: 100%;
}
</style>
<script src="leaflet-tilelayer-colorizr.js" ></script>
<meta name="theme-color" content="#000000" />
<meta
name="description"
content="Web site created using create-react-app"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>React App</title>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>
{
"short_name": "React App",
"name": "Create React App Sample",
"icons": [
{
"src": "favicon.ico",
"sizes": "64x64 32x32 24x24 16x16",
"type": "image/x-icon"
},
{
"src": "logo192.png",
"type": "image/png",
"sizes": "192x192"
},
{
"src": "logo512.png",
"type": "image/png",
"sizes": "512x512"
}
],
"start_url": ".",
"display": "standalone",
"theme_color": "#000000",
"background_color": "#ffffff"
}
# https://www.robotstxt.org/robotstxt.html
User-agent: *
Disallow:
.App {
text-align: center;
}
.App-logo {
height: 40vmin;
pointer-events: none;
}
@media (prefers-reduced-motion: no-preference) {
.App-logo {
animation: App-logo-spin infinite 20s linear;
}
}
.App-header {
background-color: #282c34;
min-height: 100vh;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
font-size: calc(10px + 2vmin);
color: white;
}
.App-link {
color: #61dafb;
}
@keyframes App-logo-spin {
from {
transform: rotate(0deg);
}
to {
transform: rotate(360deg);
}
}
import React from 'react';
import logo from './logo.svg';
import './App.css';
import MainPage from './components/MainPage'
function App() {
return (
<MainPage />
);
}
export default App;
import React from 'react';
import { render } from '@testing-library/react';
import App from './App';
test('renders learn react link', () => {
const { getByText } = render(<App />);
const linkElement = getByText(/learn react/i);
expect(linkElement).toBeInTheDocument();
});
import React from 'react';
import clsx from 'clsx';
import { makeStyles, useTheme } from '@material-ui/core/styles';
import Drawer from '@material-ui/core/Drawer';
import CssBaseline from '@material-ui/core/CssBaseline';
import AppBar from '@material-ui/core/AppBar';
import Toolbar from '@material-ui/core/Toolbar';
import List from '@material-ui/core/List';
import Typography from '@material-ui/core/Typography';
import Divider from '@material-ui/core/Divider';
import IconButton from '@material-ui/core/IconButton';
import MenuIcon from '@material-ui/icons/Menu';
import ChevronLeftIcon from '@material-ui/icons/ChevronLeft';
import ChevronRightIcon from '@material-ui/icons/ChevronRight';
import ListItem from '@material-ui/core/ListItem';
import ListItemIcon from '@material-ui/core/ListItemIcon';
import ListItemText from '@material-ui/core/ListItemText';
import InboxIcon from '@material-ui/icons/MoveToInbox';
import MailIcon from '@material-ui/icons/Mail';
import { Map as LeafletMap, TileLayer, Marker, Popup } from 'react-leaflet';
import useStyles from './theme'
import FormControlLabel from '@material-ui/core/FormControlLabel';
import Checkbox from '@material-ui/core/Checkbox';
import CommentIcon from '@material-ui/icons/Comment';
import ListItemSecondaryAction from '@material-ui/core/ListItemSecondaryAction';
import Radio from '@material-ui/core/Radio';
import RadioGroup from '@material-ui/core/RadioGroup';
import FormControl from '@material-ui/core/FormControl';
import FormLabel from '@material-ui/core/FormLabel';
import { FullDiv } from './style'
import "../../../node_modules/leaflet-tilelayer-colorizr/leaflet-tilelayer-colorizr.js"
import L from 'leaflet';
export default function MainPage() {
const classes = useStyles();
const theme = useTheme();
const [open, setOpen] = React.useState(false);