Form Management Module
1. Overview
1.1 Download Data Collection Forms
This involves downloading the ODK Forms to be rendered to the user. This may depend on the User’s role and the level of access, varying from user to user. It also includes a basic level of configurations needed to get the ODK Collect Module to be preset.
1.2 Render & Manage Data Collection Forms
This involves the ability to pre-fill certain forms, based on some parameters if needed. User can fill, send forms via this functionality. If forms are filled when offline, you can also send those forms later.
2. Setup Module In Starter App
Note: Please note that,before you start integrating the module into your project, please do get a run thorugh the demo project downloaded from the github repository, to understand the user flow.
This section lists down all the possible configuration related steps to integrate ODK, the core opensource library being used for data collection, into your starter application.
2.1 Retrieve ODK Code
2.1.1 Setup Github project
Unzip the github project to a folder. You can find the github repository at this link. Download it as zip locally and then unzip the root directory.
2.1.2 Integrating with Main App Project
Open the main project on Android Studio where you are to integrate these modules.
2.1.3 Adding commons/customworkmanager modules
If you have not customworkmanager/commons module in the project, you would need to integrate these first sequentially, to integrate the collect_app later, using the following steps.
2.1.4 Adding collect_app module (Applicable for any android module, including commons and customworkmanager, if you want to import)
Note: This is applicable for any android module, you want to import.
- Import the library module to your project (the library source becomes part of your project). Click File > New > Import Module
- Select the source directory of the Module you want to import and click Finish. The library module is copied to your project, so you can actually edit the library code.
- Open the Dependencies tab.
- Click the (+) icon and select Module Dependency. Select the module and click Ok.
- Open your build.gradle file and check that the module is now listed under dependencies.
- Sync your gradle. Clean your project.
In case you face dependency resolution errors, please see the downloaded project's main app and project gradle to see what dependencies you are missing.
2.1.5 Integrating other required modules.
Please follow the same steps for the integration of collect_app. Please checkout this link to see a more ealborate process of including libraries to your android project.
2.1.6 Syncing Gradle and Building Project.
Open the app module's build.gradle file and add a new line to the dependencies block as shown in the following snippet:
2.1.7 Modifying settings.gradle
Make sure the library is listed at the top of your settings.gradle file, as shown here for a library named ':collect_app'.
2.1.8 Adding Configuraton data
This is really important, otherwise your project won't be synced properly. Copy the config folder from the downloaded project and add to the root of your to be implemented project.
2.2 Setup ODK Aggregate
ODK Aggregate is an open source Java application that stores, analyzes, and presents XForm survey data collected using ODK Collect. It supports a wide range of data types, and is designed to work well in any hosting environment.
With Aggregate, your team can:
- Host blank XForms used by ODK Collect or other OpenRosa clients
- Store and manage XForm submission data
- Visualize collected data using maps and simple graphs
- Export and publish data in a variety of formats
Aggregate can be hosted on cloud providers such as DigitalOcean, and Amazon Web Services, or your own local or cloud server. There's also a pre-configured virtual machine image that is ready to deploy on any computer.
Please check this link to find how to set up ODK Aggregate?
Refer this link to find steps to use ODK.
2.3 Setup ODK Central
ODK Aggregate is an ODK server alternative. Like ODK Aggregate, it manages user accounts and permissions, stores form definitions, and allows data collection clients like ODK Collect to connect to it for form download and submission upload.
ODK Central is easier to install, easier to use, and more extensible with new features and functionality both directly in the software and with the use of our REST, OpenRosa, and OData programmatic APIs.
ODK Central Features:
- User accounts and management
- Role-based user permissioning
- Projects to organize users, permissions, and forms
- Form and submission upload and management - With support for form version updates - With drafts and testing on initial creation, and on version updates - With form and submission multimedia or data attachments - With a table preview of submission data
- Encrypted forms (self-supplied or project managed keys)
- OData live data feed for analysis with tools like Excel and Power BI
- Integrated checklist-based help system
- Automatic, encrypted off-site data backups to Google Drive
- Clean and modern REST API for integration and extensibility
- High performance on low-cost hardware or cloud providers
- ODK Briefcase-compatible data output
- ODK Briefcase push/pull support
Central can be hosted on cloud providers such as DigitalOcean, and Amazon Web Services, or your own local or cloud server. There's also a pre-configured virtual machine image that is ready to deploy on any computer.
Please check this link to find how to set up ODK Central?
Refer this link to find steps to use ODK.
2.4 Give Storage Permissions
2.4.1 Modifying AndroidManifest.xml
Integrating the ODK Module in your app project, would reqire to add certain user permissions. Add the following snippet in your AndroidManifest.xml
2.4.2 Storage Permission Request
In order to set up ODK, in your app, you will have to give Storage permissions. If not already incorporated you can incorporate this code in your main app module.
2.5 Initiate Module Contract
2.5.1 Initialise Collect Module From your Application Class
Please note that in order to make the modules work properly, please call this method in your Application class's onCreate() method. Please refer MyApplication.java for the same.
2.5.2 Setting File Import
Add settings.json, in the res/raw folder of your main app module. This file contains all the configurations wit reference to the integration of ODK features in your application. You will have to configure the ODK first by downloading ODK App from Play store and configuring as per the steps mentioned in this link. Please replace server_url, username, password fields in the settings.json file with your own credentials configured from the ODK app.
2.5.3 Invoke the Initialiser for the Contract
In the onCreate() of your Application-level class, after you have successfully initialised the module,please add the following method invocation.This will initialise the contract through whch you will interact with all the helper methods of the Form Module. The contract is always a work in progress, we keep adding more cases as we come across.
The method signature of registerFormManagementPackage() is as follows:
The method signature of setODKModuleStyle() is mentioned below
2.5.4 Creating Storage Directories
Please refer to 2.4.2 to see, that once you have gained storage permission access, please create storage drectories to store the media/forms/form entries in the device storage.
2.5.5 Aplying Setting file integrated in main app project
Apply the settings file configured above using the following snippet. Please do it as soon as you launch your app, preferably in your Splash Activity.
2.5.6 Resetting the forms/Data
If you are downloading the ODK Forms everytime with your session, we recommend to invoke this method when you launch your app, after settings have been applied. Use the following method:
This will delete a) Saved forms (instances folder, instances database); b) Blank forms (forms folder, forms database, itemsets database); c) Form load cache (.cache folder)
In case you just want the blank forms to be removed, use the following method invocation.
In case, a new user logs in and you want to ODK Preference data to be removed as well, use this method invocation:
2.6 Use Module Helper Methods
Please refer the sub-sections to find out the various functionalities, this module wrapper provides.
2.6.1 Delete all the previous ODK related data
Refer 2.5.6. getIFormManagementContract() is the object of IFormManagementContract registered in the Application class above. You can pass it to the various Activities via Dagger (dependency injection), or via a helper class.
2.6.2 Delete current users' form submissions
Refer 2.5.6. If in case if you just want to delete already filled/submitted forms for the user, use the following method
2.6.3 Download data collection forms
Generally, the forms are downloaded for a user based on User access/role. You could use own APIs to fetch which forms to be downloaded for a user or you could use FirebaseRemoteConfig to get the name and ID of forms to be downloaded for a user.
First, the method which can be used to fetch the list of forms available at your ODK aggregate is ahead.
FormListDownloadListener is a listener to listen to the response of the download task, with callbacks mentioned ahead.
You can then fetch specific forms from the list if fetched successfully fetched, ahead in the following manner.
getIFormManagementContract().downloadODKForms(new FormDownloadListener(), formsToBeDownloaded);//formsTobeDownloaded is a HashMap<String, FormDetails> with Key = Form ID for the form to be downloaded and Value = Form Details of the form to be downloaded, you will receive it from the List of forms downloaded
FormDownloadListener is a listener to listen to the response of the download task, with callbacks mentioned ahead.
2.5.4 Retreive Form IDs
You can fetch a form’s ID if you know the form name. This is useful in case you want to open a specific form. java
2.5.5 Pre-fill information in forms
You can prefill certain details into a form if you know the tag to be prefilled and the form’s name, in the manner mentioned ahead.
2.5.6 Launch specific forms
You can launch a specific form to edit and further send using the following invocation, given you know the name of the form.
2.5.7 View all downloaded forms
You can launch a view showing all the downloaded forms using the following invocation, where the user himself can select whichever form to fill and send.
2.7 Dynamic Cascading Dropdowns in ODK Forms
You can add cascaded dropdowns dynamically in ODK Forms, for which you can generate data dynamically, during run time. But you need to keep certain points in mind, while configuring the form and development.
- Keep in mind the name of the media file, .csv file you have stored in the forms, while configuring forms. You need to edit the same file at run time.
- The format of the .csv file is similar to this
list_name | student_name | student_label | class_key | class | section_key | section |
---|---|---|---|---|---|---|
students | Saif_Khan | Saif Khan | class6 | class6 | C | C |
students | Ram_Gautam | Ram Gautam | class1 | class1 | B | B |
The dropdowns work only until 3 levels of hierarchy, ie. A>B>C, here Class>Section>Student
You need to know the keys for which you need to fill the data, Here they are student, class and section. This is so because your form is coupled with these keys
Names can't contain spaces. List_name would be same for all the rows.
Using this
- First fetch all the forms, which have media folders with the file with the name you want to replace after editing. Use the following method. Please invoke this method only after forms have been downloaded, else,you won't get correct output.
referenceFileName is the name of the .csv file you want to find and then edit, eg. student_list.csv
- If the value(ArrayList) returned by the above method call, has size > 0, which means some of the forms have this same media file attached with them. The following method will edit this file and then save later.
Here, Important thing to note is that the JSON Array should have keys in format as the .csv file. Refer the following as sample resource:
Here, section, student,class match with the keys as from the .csv file.
Please note that, what we essentially do is:
- Check if media files exist in those folders, if not, create a copy and replace that in actual folder.
- Check validity of keys
- Create a final list of elements, parse that to create a CSV
- Save the CSV in that format in the destination folder
3. FAQs
3.1 What version of ODK Collect we are using?
We are using v1.26.1.
3.2 What target version of Android we are using?
API 28. Please build your app for that version only. We will upgrade it to API 29 soon.
3.3 How can I upgrade the ODK Collect verion?
Please check this link.. Download the latest version. Check the changes made in the latest release. Override the changes made in the latest versions.
3.4 My application crashes due to permission exceptions. What can I do?
Please note that, storage permissions though have been asked from user at the launch of app, but when launching the form view, you will have to provide, microphone/location permissions, if your forms contain questions containing media/geo-location.
Please refer the table below.
Permission | Reason |
---|---|
Storage | required for Collect to be able to access and save form data |
Camera | required by image and video questions to capture new media |
Contacts | required to configure a Google account for Google Drive, Google Sheets submissions |
Location | required for location questions |
Microphone | required by audio and video questions to capture new media |
Phone | optional on form send to include deviceID in the submission and required for forms that capture device metadata |
3.5 My build is failing. What should I do to debug?
Here are the approaches you could follow; a) Please sync your gradle and clean your project. b) Check for dependency resolution errors, check the downloaded roject to see the type and version of libraries imported. c) Check your google-services.jsn, it should be compatible with the applicationId mentioned in your project build.gradle
3.6 My forms don't seem to behave in the way they are supposed to be. What should I do?
Please check the same forms that you have configured on XLS, on the ODK Collect app, if the form doesn't work there, please check your form structure, you will have to debug to check as this means there has been some issue with your form.
3.7 How can I configure auto-sending the forms on internet connectivity issue?
ODK is a robust tool developed to handle offline scenarios. Auto send When enabled in setings.json (Add "autosend": "on"), forms are sent immediately when they are finalized, if the device can connect to the internet. If an internet connection is not available at the time of finalization, your finalized forms will be queued to send as soon as connectivity is established. You can specify whether to send over WiFi, cellular data, or both.
Feel free to write into tech@samagragovernance.in in case you have questions, feedback or want to know more!
4. Coming Soon
Please review the following section to get information about planned updates to this module.