Heliotrope

Code:

Thanks to Skylight:

Jump In: Issue Tracker

Table of Contents

• What is Fulcrum? What is Heliotrope?
  • Feature List
• Help/Contact
• Getting started
  • Prerequisites
  • Initial setup
• Debugging
• Testing
• License
• Contributing
• Ancillary Repositories
• Acknowledgements

What is Fulcrum? What is Heliotrope?

Fulcrum is an in-development publishing platform and set of hosting and publishing services aimed to help scholarly publishers present the full richness of their authors' research outputs in a durable, discoverable, and flexible form. Its first phase of development is focused on providing branded companion websites for books. Now, in its second phase, it will expand to host complete e-book collections, journals, and new forms of multimodal publications as well as provide a set of hosting and publishing services for publishers.

Heliotrope is the codebase behind the Fulcrum publishing platform. It is a Rails application that extends Hyrax–the popular front-end repository solution from the Samvera open source community–to provide solutions for scholarly publishers.

It is built by the University of Michigan Library and managed by the Library's publishing division, Michigan Publishing.

Feature List

In addition to the features that come with Hyrax, Heliotrope offers the following:

• Web-based e-book reader for FileSets that are valid EPUBs. The e-book reader is delivered as a single-page JS application that is included with Heliotrope as a gem. See the wiki for more details and features specific to the e-book reader. (Example)
• Publisher catalog page listing Works associated with publisher (Example).
• Customized branding (logo, colors, fonts) for publisher that is applied to associated Works and FileSets (Example).
• Publisher-specific usage analytics (eg, support for multiple Google Analytics IDs on a single page, allowing each publisher to have their own Google Analytics property).
• Bulk importer for Works and their children.
• Embed codes for FileSets, allowing FileSets to be embedded into external websites and EPUB files.
• Support for FileSets that are hosted externally (Example).
• Additional metadata fields for Books and their associated media.
• Accessibility improvements to meet Section 508 and WCAG 2.0AA guidelines.
• Delivery of time-based media through an accessible media player, AblePlayer. Support for transcripts and captions when provided (Example).
• Delivery of IIIF-served images using Leaflet (Example).
• Integration with Jekyll static-site generator for "aboutware" static pages and blog.

Help/Contact

For additional details and helpful hints read our ever growing Wiki. To get in touch with us over e-mail, contact the Fulcrum Developers List.

Getting started

Prerequisites

• Install and configure Hyrax prerequisites.
• Install redis
• Install mysql (Install MySQL OS X El Capitan)

Initial setup

1. Getting a local copy, bundle install gems, and execute setup script

$ git clone https://github.com/mlibrary/heliotrope.git
$ cd heliotrope
$ bundle install
$ yarn install
$ bundle exec ./bin/setup
$ bundle exec rails checkpoint:migrate
$ bundle exec rake assets:precompile

See the Wiki for information on Jekyll integration.

2. Create users

Create fulcrum-system user

There is a rails task you execute to create a "fulcrum-system" user.

$ bundle exec rails system_user

If you need to run this when the app has been deployed, execute:

$ RAILS_ENV=production bundle exec rails system_user

Make yourself a "platform" admin

There is a rails task you can execute to create a "platform" admin user. It will prompt you for an email address and then create a user with the correct role.

$ bundle exec rails admin

If you need to run this when the app has been deployed, execute:

$ RAILS_ENV=production bundle exec rails admin

3. Run the application

Execute this command to start Fedora, Solr and Rails servers:

$ bundle exec rails hydra:server

Or, if you prefer to start each server individually execute each of the following commands in a seperate shells: (you must use this alternate option if running on a VM)

$ redis-server /usr/local/etc/redis.conf
$ fcrepo_wrapper -p 8984 --no-jms
$ solr_wrapper -p 8983 -d solr/config/ --collection_name hydra-development
$ bundle exec rails s

NOTE: You'll also want to make sure that you have MySQL started.

NOTE: There are also config files available for running the wrappers (which save you from having to remember ports, collection names etc). Their settings attempt to persist your Solr index as you move between dev and test. Use like so:

$ fcrepo_wrapper --config .wrap_conf/fcrepo_dev
$ solr_wrapper --config .wrap_conf/solr_dev

4. Create default administrative set

$ bundle exec rails hyrax:default_admin_set:create

Docker Setup for Development

Instead of running locally with solr_wrapper and fcrepo_wrapper, everything can be run with docker compose. The Docker setup uses entrypoints under docker/entrypoints/. On startup, the app container will wait for dependencies and then run setup tasks (including db:setup/db:migrate, checkpoint:migrate, and system_user) automatically.

Use bin/compose for Docker Compose commands in this repo on both macOS and Linux.

• bin/compose automatically sets DOCKER_DEFAULT_PLATFORM=linux/amd64 on macOS Apple Silicon unless you already set it.
• bin/dc remains available as a compatibility alias to bin/compose.

1. Create required config files (first time only)

config/database.yml and config/secrets.yml are not committed to the repo. Copy the samples before starting the stack:

cp config/database.yml.sample config/database.yml
cp config/secrets.yml.sample config/secrets.yml

2. Build and start services

bin/compose up -d --build

For EPUB search compatibility with legacy SQLite databases that use FTS4, configure Bundler before install:

bundle config build.sqlite3 "--with-sqlite-cflags=-DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4"
bundle config force_ruby_platform true

For Capistrano deployments, run the same bundle config build.sqlite3 ... command on the app release before bundler:install in your deploy project.

This starts the main development stack:

• app (Rails on http://localhost:3000)
• resque
• db (MariaDB)
• solr
• fcrepo
• redis

3. Create a platform admin (one time per database)

bin/compose exec app bundle exec rails admin

4. Create default administrative set (if needed)

bin/compose exec app bundle exec rails hyrax:default_admin_set:create

5. Stop services

bin/compose down

Debugging

Explain Partials

When running Rails server, set the EXPLAIN_PARTIALS environment variable to show partials being rendered in source html of your views. You can view this info using your browser's inspect element mode.

$ EXPLAIN_PARTIALS=true bundle exec rails s

NOTE: Because this feature can add a fair bit of overhead, it is restricted to only run in development mode.

Testing

To execute the continuous integration task run by CircleCI

$ bundle exec rails ci

Starting servers individually when testing

Alternatively, you can start up each server individually. This may be preferable because the ci task starts up and tears down Fedora and Solr before/after the test suite is run.

1. Start up FCrepo

$ fcrepo_wrapper -p 8986 --no-jms

or

$ fcrepo_wrapper --config .wrap_conf/fcrepo_test

2. Start up Solr

$ solr_wrapper -p 8985 -d solr/config/ --collection_name hydra-test

or

$ solr_wrapper --config .wrap_conf/solr_test

3. Run tests

$ bundle exec rails rubocop
$ bundle exec rails ruumba
$ bundle exec rails lib_spec
$ bundle exec rspec

Running tests with Docker

Use the Docker Compose test profile for test Solr/Fedora and a dedicated test container.

1. Start test infrastructure

bin/compose --profile test up -d db redis solr-test fcrepo-test

2. Run the full test suite

bin/compose --profile test run --rm test

3. Run individual checks

bin/compose --profile test run --rm test bundle exec rspec spec/path/to_spec.rb
bin/compose --profile test run --rm test bundle exec rubocop
bin/compose --profile test run --rm test bundle exec rails ruumba
bin/compose --profile test run --rm test bundle exec rails lib_spec

4. Stop test infrastructure

bin/compose --profile test stop solr-test fcrepo-test

System specs

System specs may be skipped in CircleCI due to timing-related instability. This won't affect running rspec locally either directly or through the ci task.

Running specs individually

To run individual specs located in the ./lib/spec directory (a.k.a lib_spec) first step into the lib directory and then execute rspec.

$ cd lib
$ bundle exec rspec

Special note on running tests

As of June 20, 2017 there are tests that require the static pages to be built in order for routing to happen correctly (See Static Pages and Blog documentation). This means you need to execute

$ bundle exec rails jekyll:deploy

before running rspec. This only need be executed once. If you followed step 1 of the initial setup then you did this already.

License

Heliotrope is available under the Apache 2.0 license.

Contributing

We'd love to accept your contributions and there are lots of ways to engage with the Fulcrum project and Heliotrope codebase even if you aren't a developer. Please see our contributing guidelines for how you can get involved.

Please note: As of May 14 2018, we have migrated our issues to a JIRA project and are no longer creating new issues in GitHub. All GitHub issues that were open at the time of migration will be updated with relevant links to the corresponding JIRA issue. If you find bugs or would like to open an issue, you can still use GitHub for that to open the conversation with Fulcrum developers.

Ancillary Repositories

The Fulcrum Development Team have created a number of ancillary code repositories for handling different areas of business logic related to our use of the Heliotrope repository software. None of the libraries listed below are required in any way for the operation of Heliotrope and are tuned to the use cases and business logic relevant to the Fulcrum instance. They are offered here for possible adaptation and repurposing, or just to illustrate the cluster of tasks that necessarily arise around the operation of repository publishing software.

• Heliotropium is a collection of miscellaneous scripts that provide support and infrastructure for heliotrope in the Fulcrum production environment but don't require direct access to Hyrax data structures. Examples include task scheduling and fetching MARC records.
• Winterberry provides a number of scripts for processing content in a production stream for eventual ingest into Heliotrope.
• Greensub supports a number of business processes related to restricted repository content: assigning Monographs to specific restricted Products, and authorizing Institutions and Individuals to access those Products.
• Turnsole provides the RESTful API for Greensub to communicate with Heliotrope.

Acknowledgements

Initial development has been supported by a generous grant from the Andrew W. Mellon Foundation and implemented by the University of Michigan Library and Press working with partners from Indiana, Minnesota, Northwestern, and Penn State universities and Data Curation Experts.